The probability is the outcome when you integrate mathematically over the probability density function (PDF). As you pointed out, taking only a point of this PDF does not really make sense since you cannot really interpret a single point here in a reasonable way (besides to be a literally infinitesimal small intervalโฆ as you pointed out, too!).
But if you integrate over a defined interval of the PDF you can derive a tangible probability, which you can interpret, e.g. in the example of an ROC analysis or when evaluating false positives etc., see also:
-
https://upload.wikimedia.org/wikipedia/commons/4/4f/ROC_curves.svg
-
Anomaly Detection: How to improve? - #2 by Christian_Simonis
Please let me know if this helps, @Numair!
Best regards
Christian