HLP for sensor data?

I place some sensors in the production line to monitor the quality of the manufacturing process and predict each workpiece is qualified or not. The ground truth is determined by inspection equipment and that can output a quality value(hope the value is higher than the specific threshold). I would like to know the following -

Suppose even a human inspector looking at the time series sensor data cannot tell if it’s qualified or not. How could I measure HLP and do the error analysis?

Thank you

When it’s hard to get a reasonable baseline, something like this can be used since a model doing the dumbest thing can achieve a minimum level of performance.

Hello Bru,
Normally, you can find many research papers which try to address your problem. If not exactly, some other simplified version of your problem can always be found. You could try to implement those or if the paper’s repository is available you could use their solution as the baseline. Afterwards, you can fine-tune the model or try to enhance the model given your needs. That way, you potentially could beat your baseline and provide something valuable to the research/industrial community.
I hope that helps.
All the best,