All prior assignment cells passed tests including: covariance_matrix_from_examples, get_top_magnitude_indices, and remove_from_list.
After working on get_top_covariances over two days and reviewing all the relevant Topics here, I ask for feedback on approaches to the following assertion error:
The values for relevant_indices and highest_covariance are:
In words, I have assigned to relevant_indices the np.indices’d flattened output of the covariance matrix from examples applied to classification changes.
I then have assigned to highest_covariance the top_n elements of get_top_maginitude_indices applied to relevant_indices after remove(ing)_from_list the target_index.
Fun fact: I’m taking a TensorFlow course here also and realized that NumPy, TensorFlow, and Pytorch all have different flatten functions/methods–variety, confusion, and fun all around are the spice of life!!!
LATER ADDITION: As I was chopping vegetables for dinner, I realized that it is the output of relevant indices that I’m perplexed by–the output of the relevant indices SHOULDN’T be as displayed but I’m not figuring out how to get the “original” indices. Is there more that I’m missing? [Sadly for you all, I find that once I’ve posted a Topic my brain is much more on task working on the the solution without me consciously working on it–the chopping vegetables is a typical example.]
Hi, Jeff.
Hmmm, that doesn’t sound like it maps to how I implemented this, but maybe I’m just not thinking hard enough about your description. The steps use the previously implemented functions:
You compute the covariance matrix by calling covariance_matrix_from_examples.
Then you extract the indices with the highest values (using get_top_magnitude_indices).
Then you remove the target index from that list with the remove_from_list function.
I added some print statements to see more info and here’s my output from running that test cell.
covary.shape = (4, 4)
covary =
[[10.05051235 2.0828533 -0.48089952 -4.97268808]
[ 2.0828533 10.89214084 5.0277819 3.98804526]
[-0.48089952 5.0277819 9.98912851 2.14735945]
[-4.97268808 3.98804526 2.14735945 11.02674517]]
values [ 2.0828533 10.89214084 5.0277819 3.98804526]
top_indices = [1 2 3 0]
all_indices.shape = (4,)
relevant_indices.shape = (3,)
highest_cov.shape = (3,)
relevant_indices [2 3 0]
highest_covariances [5.0277819 3.98804526 2.0828533 ]
covary.shape = (4, 4)
covary =
[[10.05051235 2.0828533 -0.48089952 -4.97268808]
[ 2.0828533 10.89214084 5.0277819 3.98804526]
[-0.48089952 5.0277819 9.98912851 2.14735945]
[-4.97268808 3.98804526 2.14735945 11.02674517]]
values [10.05051235 2.0828533 -0.48089952 -4.97268808]
top_indices = [0 3 1 2]
all_indices.shape = (4,)
relevant_indices.shape = (3,)
highest_cov.shape = (3,)
relevant_indices [3 1 2]
highest_covariances [-4.97268808 2.0828533 -0.48089952]
covary.shape = (4, 4)
covary =
[[10.05051235 2.0828533 -0.48089952 -4.97268808]
[ 2.0828533 10.89214084 5.0277819 3.98804526]
[-0.48089952 5.0277819 9.98912851 2.14735945]
[-4.97268808 3.98804526 2.14735945 11.02674517]]
values [-0.48089952 5.0277819 9.98912851 2.14735945]
top_indices = [2 1 3 0]
all_indices.shape = (4,)
relevant_indices.shape = (2,)
highest_cov.shape = (2,)
relevant_indices [1 3]
highest_covariances [5.0277819 2.14735945]
covary.shape = (4, 4)
covary =
[[10.05051235 2.0828533 -0.48089952 -4.97268808]
[ 2.0828533 10.89214084 5.0277819 3.98804526]
[-0.48089952 5.0277819 9.98912851 2.14735945]
[-4.97268808 3.98804526 2.14735945 11.02674517]]
values [-4.97268808 3.98804526 2.14735945 11.02674517]
top_indices = [3 0 1 2]
all_indices.shape = (4,)
relevant_indices.shape = (2,)
highest_cov.shape = (2,)
relevant_indices [0 1]
highest_covariances [-4.97268808 3.98804526]
All tests passed
Actually notice that my covariance matrix is already different than what you show. Are you sure that all your previous functions pass their unit tests?
Thanks for the feedback, Paul!
My code had passed all the previous tests, but as you noted I was getting a different covariance matrix than you did (and what I was changing wasn’t helping =), I downloaded a clean version of the notebook and started anew. The first run and unit test of the covariance matrix output now matches yours:
BUT, if that code cell and the unit test are run again, the covariance matrix output changes:
So, there must be some seed setting that works the 1st time a notebook is run, but not subsequent ones.
Also, I implemented it the same way you described but tried to write my description so I wasn’t “sharing code” here and so it was written in a not straight forward fashion…oh well.
But I have implemented it as noted in Optional hints 3) It may be easiest to solve this if you find the relevant_indices first, and then use relevant_indices to calculate highest_covariances.
I’m still stumbling over understanding the relevant_indices: the covary matrix is 4 x 4, so I using ravel to get it into a “flat” array, then argsort in descending order [::-1] the absolute value of the array. That’s my relevant_indices. As usual, I have a big BUT: the argsort is not in descending error:
Not sure what to try next…trying to figure out how to get the indices of a 2D array in descending order (without raveling/flattening it)…
Your mistake is that you are not supposed to flatten the whole covariance matrix: you are supposed to select the row of it that corresponds to the requested target index, right? That is probably the crux of all your issues here.
BTW it looks like you’re right that there is something stateful about this (probably the random seeds) that causes the results to change if you run the test cell more than once. But if you do “Cell → Run All”, then it looks like you do get consistent results.
Thanks for the guidance Paul!!!
I believe it is my not yet understanding of the all assignment’s details that trip me up.
I am getting an AssertionError on the final code cell but C2W2: All Tests Passed, But Still Getting Errors in The Last Cell - #6 by Nithin_Skantha_M describes this as not critical and due to the nature of the calculations.
I’ve submitted the assignment and am hoping for the best!
Take care 
Hello everybody,
I have the same problem. I have been trying different solutions for the last three days, and I always get the same message: My [8] value is -1.240295609191389.
Which cell is that? Please show us the full output. Note the link in the previous post on this thread: you can pass the grader even if you get an assertion error in the final code cell.
The problem occurs in the last cell. This is the message I received:
AssertionError Traceback (most recent call last)
Input In [16], in <cell line: 5>()
3 assert relevant_indices[9] == 34
4 assert len(relevant_indices) == 10
----> 5 assert highest_covariances[8] - (-1.2418) < 1e-3
6 for index, covariance in zip(relevant_indices, highest_covariances):
7 print(f"{feature_names[index]} {covariance:f}")
I was afraid to submit my assignment, but I received a grade of 100%. However, I would still like to know what happened and what values would be received if this error didn’t occur.
