Practice lab 2, on anomaly detection
To be honest, this is a level of precision I have never dealt with in an error that is not actually an integer. If this F1 error were an integer, I could round or truncate or take the ceiling. But it’s not.
I have done some extremely fine detail work in porting FORTRAN to C++ for a general relativity code, however, truncation and roundoff error after a few thousand or more steps eventually did accumulate as a discrepancy between the two codes, so the finest match I was able to implement for the black hole orbits was about 10^-11 to 10^-13 relative error. It was our belief that was roundoff error based on a truncation error analysis of the convergence of the discontinuous galerkin, Runga Kutta, and multipole moment modes and orders. Although the original intent was to use the C++ code to parallelize in HPX, we eventually shifted to comparing the code to a collaborators frequency domain code, which involved a python error analysis.
HOWEVER, I’m not totally sure what to do with the distinction between 0.9999999999999999 and 1 in python, for a number that is fundamentally a float, as the F1 statistic is.
If it were 0.9 repeating in mathematics, the two numbers would be literally equal, but clearly they’re not quite, since there is in fact a digit where the 9 stops repeating. The difference is obviously pretty small.
The place where I’m seeing this is in the select_threshold graded function.
Best epsilon found using cross-validation: 8.990853e-05
Best F1 on Cross Validation Set: 0.875000
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-43-d487302cf8bb> in <module>
6
7 # UNIT TEST
----> 8 select_threshold_test(select_threshold)
~/work/public_tests.py in select_threshold_test(target)
8 best_epsilon, best_F1 = target(y_val, p_val)
9 assert np.isclose(best_epsilon, 0.04, atol=0.3 / 1000), f"Wrong best_epsilon. Expected: {0.04} got: {best_epsilon}"
---> 10 assert best_F1 == 1, f"Wrong best_F1. Expected: 1 got: {best_F1}"
11
12 y_val = np.array([1] * 5 + [0] * 25)
AssertionError: Wrong best_F1. Expected: 1 got: 0.9999999999999999
I feel like I probably did compute the values correctly, since my output essentially matches the expected output… and I suppose given that multiple methods are allowed for solving this problem, it’s completely possible to have answers with small variations due to roundoff error. However, I’m not totally sure how to fix this so that it passes the autograder.
I think I could write a short section rounding it to 3 significant figures as shown in the output, which would correct the roundoff error in the 0.9999999999999999 to the 1 (which is what the autograder asks for), but I’m not sure whether or not that’s the intended method. Could someone please clarify?
Thanks,
Steven
