Dinosaurus_Island_Character_level_language_model np.random.choice

Course Q&A Deep Learning Specialization Sequence Models
1

In the programming exercise,
in the sample function [def sample(parameters, char_to_ix, seed): … return indices],

Step 3: Sample the index of a character within the vocabulary from the probability distribution y

    # (see additional hints above)
    idx = np.random.choice(range(len(y)), p = np.ravel(y))

I think idx return the exact constant value in the depends on the y distribution, when the seed is set as a definite value.

But in instructions said below as:

  • Sampling is a technique you can use to pick the index of the next character according to a probability distribution.
    • To begin character-level sampling:
      • Input a “dummy” vector of zeros as a default input
      • Run one step of forward propagation to get 𝑎⟨1⟩ (your first character) and 𝑦̂ ⟨1⟩ (probability distribution for the following character)
      • When sampling, avoid generating the same result each time given the starting letter (and make your names more interesting!) by using np.random.choice

how come in previous function, idx is set to a specific value according to the y distribution, and the function passed the test?

1 Like
2

I am not sure if I understand your question.

In training, we always want to use the next letter in the training sequence.

In making predictions, we want to use a random selection of the outputs, so that we don’t always get the same result. In this assignment we’re trying to create a number of different dinosaur names - not always the same one.

3

It is a good point that they set the random seeds in pretty much every assignment here, but that is just to make it easier to write test cases and the graders. You normally wouldn’t do that in a real application.

But notice that they set the seed in the “for” loop in sample and they set it to a different value each loop:

        # For grading purposes
        np.random.seed(counter + seed)

That means the overall sequence of results will be predictable, but not every name will be the same.

You can run experiments with any of the random functions to get a sense for how this all works. With PRNG functions, setting the seed to a particular value starts the sequence at a particular point. Then subsequent calls will also be predictable, but different each time. Write some sample code like this:

for ii in range(10):
    np.random.seed(42)
    print(f"ii = {ii}: {np.random.randint(0,10,(1,8))}")

That gives you the same result every time:

ii = 0: [[6 3 7 4 6 9 2 6]]
ii = 1: [[6 3 7 4 6 9 2 6]]
ii = 2: [[6 3 7 4 6 9 2 6]]
ii = 3: [[6 3 7 4 6 9 2 6]]
ii = 4: [[6 3 7 4 6 9 2 6]]
ii = 5: [[6 3 7 4 6 9 2 6]]
ii = 6: [[6 3 7 4 6 9 2 6]]
ii = 7: [[6 3 7 4 6 9 2 6]]
ii = 8: [[6 3 7 4 6 9 2 6]]
ii = 9: [[6 3 7 4 6 9 2 6]]

Now try this:

np.random.seed(42)
for ii in range(10):
    print(f"ii = {ii}: {np.random.randint(0,10,(1,8))}")

ii = 0: [[6 3 7 4 6 9 2 6]]
ii = 1: [[7 4 3 7 7 2 5 4]]
ii = 2: [[1 7 5 1 4 0 9 5]]
ii = 3: [[8 0 9 2 6 3 8 2]]
ii = 4: [[4 2 6 4 8 6 1 3]]
ii = 5: [[8 1 9 8 9 4 1 3]]
ii = 6: [[6 7 2 0 3 1 7 3]]
ii = 7: [[1 5 5 9 3 5 1 9]]
ii = 8: [[1 9 3 7 6 8 7 4]]
ii = 9: [[1 4 7 9 8 8 0 8]]

Starts the same as the previous case because of the same seed value, but then different each time, right? Then try a method similar to the one used in the sample function by changing the seed each iteration:

for ii in range(10):
    np.random.seed(42+ii)
    print(f"ii = {ii}: {np.random.randint(0,10,(1,8))}")

ii = 0: [[6 3 7 4 6 9 2 6]]
ii = 1: [[4 0 1 5 0 3 1 2]]
ii = 2: [[4 3 1 3 0 4 3 8]]
ii = 3: [[3 0 5 3 4 9 8 1]]
ii = 4: [[5 8 4 3 2 0 2 8]]
ii = 5: [[7 6 7 8 8 3 0 7]]
ii = 6: [[0 3 1 4 0 0 6 6]]
ii = 7: [[8 5 6 4 2 5 5 1]]
ii = 8: [[0 0 1 4 6 5 6 6]]
ii = 9: [[9 5 0 9 5 0 5 4]]

Also different each iteration but in a different way. :nerd_face:

Of course the high level point is that if you run all of those code segments again, the results are exactly the same every time, meaning that the sequences are predictable once you’ve seen them the first time, which is the whole reason for using the seeds in the case that you’re trying to write test cases and graders.