So, I created the schema display and calculation function but I didn’t get any anomaly. I downloaded the dataset and explored it with pandas and found the exactly value what was supposed to be missing.
This is the error I got and pandas code
Hi @Sodipe_Paul
Thanks for your question.
It is a bit hard to say where the problem lies from the info given, but I suggest you check the following;
Please check the schema generated from the train_stats at exercise 3 and you should see no Neurophysiology value in the Medical Specialty domain, like in the snip below.
If you do see Neurophysiology there, please check if you called the right stats in generating the schema and in your calculate_and_display_anomalies function. You can restart the kernel again before checking the code to make sure that when checking the schema at the training stage, it was not overwritten by the schema updates later on in the notebook.
you can also call the tfdv.display_schema(schema=schema) at different stages of your notebook to see where the schema got updated with the eval stats.
Hope this helps finding the source of the problem.
Happy learning !
Maarten
Thanks, Maarten.
I finally found what I was doing wrong.