Faced similar problems, solved by carefully going through this thread. On a more general note, I found (as many others did too) that restarting the kernel was essential. This is not good, in my opinion, and may be looked at as a future improvement area. Thank you.
5 Likes
@SaurabhSawhney : thank you for the tip
Same realization for me, you have to restart the kernel after your modifications otherwise your mods can be correct but the execution fails until you restart the kernel.
2 Likes
Yeah. Same here. I would never have thought of restarting the kernel and running all cells to fix the problem… that’s a flaw in the notebook. Otherwise the same problem with argmax axis.
3 Likes
I use x = tf.math.argmax(out, axis=-1) but I get the following error: WARNING:tensorflow:Functional inputs must come from tf.keras.Input (thus holding past layer metadata), they cannot be the output of a previous non-Input layer. Here, a tensor specified as input to “functional_4” was not an Input tensor, it was generated by layer repeat_vector_149.
Note that input tensors are instantiated via tensor = tf.keras.Input(shape).
The tensor that caused the issue was: repeat_vector_149/Tile:0
What exactly is the problem? I’ve been stuck with this for many days 
2 Likes
Hi @AliEser
x = tf.math.argmax(out, axis=-1)
is fine. There must be something else that caused the error. Could you post the error trace so that we have more information to work with.
1 Like
Did you set it to -1?? Its not working for me…
1 Like
Did you restart the kernel after you change your code? I met the same issue as you after set axis = -1, but after I restart the kernel and rerun, all tests passed. It’s ridiculous.
3 Likes
After you reset it, restart the kernel and rerun. It would work. I think this notebook has some issues.
1 Like
They should really fix the kernel issues. I literally lost hours on this function and it was correct all along…
2 Likes
I have somewhat similar issues as many of you in this thread. More interesting the error changes sometimes even when I don’t change the code. Current error when running unit test is: "AttributeError: The layer “lstm has multiple inbound nodes, with different input shapes. Hence the notion of “input shape” is ill-defined for the layer. Use get_input_shape_at(node_index) instead.”
I went through the comments here and believe I use tf.math.argmax and tf.one_hot correctly. Next line RepeatVector(1)(x) is given to us in instructions, so it is basically copy-paste.
Step 2.A where we use LSTM_cell the input is x. However, I do believe we need to slice it. Just like in the first problem of Assignment 3 (my current issue with a second problem music_inference_model() function). If I follow the analogy of the first problem than x[:,t,:] should work. I also think we need to pass x through reshaper. But it doesn’t. The only thing that worked for me is x[None, t, :] which I don’t understand why? also, do we need to reshape x? since it is not mentioned in the instructions.
Lastly to note, I do rest kernel almost every time I run this function. Otherwise there are a whole bunch of other errors. My lab id is kgbjkxzz (for instructors).
Thank you!
1 Like
There must be an update to the assignment that informs the user that updating the axis will require a kernel restart.
Hours wasted due to a bug that could have been prevented with a simple note in the notebook to restart the kernel.
3 Likes
“Restart the kernel” doesn’t have anything specific to do with changing the axis argument.
It’s a work-around that seems to fix a multitude of ills with the notebook platform. It’s the only tool available to put the notebook into a known state.
But in general, restarting the kernel isn’t necessary.
But you do have to re-run all of the cells, especially if you have modified anything that impacts the global LSTM_cell() object.
2 Likes
Error on running music inference model() -
ValueError Traceback (most recent call last)
in
----> 1 inference_model = music_inference_model(LSTM_cell, densor, Ty = 50)
in music_inference_model(LSTM_cell, densor, Ty)
38 for t in range(Ty):
39 # Step 2.A: Perform one step of LSTM_cell. Use “x”, not “x0” (≈1 line)
—> 40 a, _, c = LSTM_cell(x, initial_state=[a, c])
41
42 # Step 2.B: Apply Dense layer to the hidden state output of the LSTM_cell (≈1 line)
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/recurrent.py in call(self, inputs, initial_state, constants, **kwargs)
707 # Perform the call with temporarily replaced input_spec
708 self.input_spec = full_input_spec
→ 709 output = super(RNN, self).call(full_input, **kwargs)
710 # Remove the additional_specs from input spec and keep the rest. It is
711 # important to keep since the input spec was populated by build(), and
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in call(self, *args, **kwargs)
924 if _in_functional_construction_mode(self, inputs, args, kwargs, input_list):
925 return self._functional_construction_call(inputs, args, kwargs,
→ 926 input_list)
927
928 # Maintains info about the Layer.call stack.
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in _functional_construction_call(self, inputs, args, kwargs, input_list)
1090 # TODO(reedwm): We should assert input compatibility after the inputs
1091 # are casted, not before.
→ 1092 input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
1093 graph = backend.get_graph()
1094 # Use self._name_scope() to avoid auto-incrementing the name.
/opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
225 ’ is incompatible with layer ’ + layer_name +
226 ‘: expected shape=’ + str(spec.shape) +
→ 227 ‘, found shape=’ + str(shape))
228
229
ValueError: Input 0 is incompatible with layer lstm: expected shape=(None, None, 90), found shape=[90, 1, 64]
1 Like
ValueError: Input 0 is incompatible with layer lstm: expected shape=(None, None, 90), found shape=[90, 1, 64]
This says that the input shape is not what LSTM_cell expects. Probably, you may need to correct one-by-one. The first step is to correct the last dimension. It needs to be equal to the shape of input values, i.e, 90. I’m afraid that you set the depth, 2nd argument, for tf.one_hot() to 64, which is the number of units in LSTM_cell.
And, it is better for you to track how the shape of x changes.
shape x: (None, 1, 90)
shape x after argmax: (None,)
shape x after onehot: (None, 90)
shape x after RepeatVector: (None, 1, 90)
For further debugging, it is better to reset LSTM_cell everytime. (Do not need to restart a kernel.)
LSTM_cell = LSTM(n_a, return_state = True)
For more detail, please visit this thread
1 Like
This is very useful.
However, earlier I went ahead and submitted the assignment with this nagging error. The grader graded it fully.
1 Like
Hello, NN3.
Restarting the kernel eliminates the unnecessary derivatives and caches that were generated by the cells while playing with the codes, and yes, of course you need to re-run the all the cells from the beginning through shift+enter key.
Upon further review, I think I understand the issue with restarting the kernel. It’s due to the way this notebook uses the global object “LSTM_cell”.
Using the wrong axis for argmax() in music_inference_model() will mangle the configuration of LSTM_cell. Since that’s a global resource that was created by the djmodel(), once LSTM_cell gets mangled, you have to go back and re-create it again.
Restarting the kernel is a sure-fire way to erase the global objects, then you have to run all of the cells again in order to re-create LSTM_cell.
Hi AllEser,
Your problem was exactly my problem. Kindly define your model properly in Step 3. As it is initializing the model, it should take the initialized inputs only i.e. x0, a0,c0 and not x0,a,c. I am sure this will help.
1 Like
Thanks for you reply, but please note that this is a pretty old thread. E.g. that post from AliEser was created almost two years ago in July of 2021.
1 Like
I know. I replied just to help anyone else facing that problem. As no one replied regarding setting up the model properly with initial values. Adds one more possible solution for someone fresh who will visit the thread in future as I myself got help from this thread just an hour ago even if it was more than 2 yrs old.
Cheers
1 Like