C5W1A2: Sample function in Character level language model - Dinosaurus Island

Course Q&A Deep Learning Specialization Sequence Models
1

I am having an error of that probabilities in numpy.random.choice does not sum up to one. BUT
my probabilites sum up to 9.9999999997 something.

These are the dimensions:

x → shape (27,1) zeros
a_prev → shape (100, 1) zeros

(And i dont understand why shapes of x and a_prev are different here)

np.random.choice(range(len(y.ravel())), p = y.ravel())
2

Why do you believe that numpy.random.choice() should sum to one?

x and a_prev should be different sizes.

  • x is the number of examples.
  • The size of a_prev relates to the number of units in the layer.
1 Like
3

Because softmax output probs right?
And all the probs should sum up to one.

But they don’t like they sum up to 9.9999997.

Thats what i need to know how to solve this i can’t figure out where the problem actually lies.

4

Softmax output probs and np.random.choice know that all the probs should sum up to one.

5

“softmax” isn’t mentioned in the instructions for this assignment.

6
  • Details about 𝑦̂ ⟨𝑡+1⟩y^⟨t+1⟩:
    • Note that 𝑦̂ ⟨𝑡+1⟩y^⟨t+1⟩ is a (softmax) probability vector (its entries are between 0 and 1 and sum to 1).
    • 𝑦̂ ⟨𝑡+1⟩𝑖y^i⟨t+1⟩ represents the probability that the character indexed by “i” is the next character.
    • A softmax() function is provided for you to use.
7

@hardikdhuri , I was stuck on this for a while too.

Use print to check the shape of your y. It should be a vector that represents your first output, so if our ‘vocabulary’ is 27 character long, it should be a vector of length 27, or shape of (27,). The probabilities should sum to 1, not 100 so if you have it summing to 99.99999997 then either your random choice is implemented wrong or your y is summing the vocabulary vector over 100 iterations (as was my case).

8

I stumbled upon this issue as well.
Despite the error that the probabilities don’t sum up to one, this is actually NOT the issue in my case. It was the shape issue with some variables that I forgot to correct/fix all along. I was curious how fixing the shape would fix the “sum not equal to 1” issue. Then I print the sum of y, and it sometimes still has a SUM of 0.99999998 or 1.00000002. But IT PASSED THE TEST REGARDLESS.
Hope this finding helps future learners.