InvalidArgumentError various model fit code tried

Course Q&A TensorFlow Developer Professional Natural Language Processing in TensorFlow
, ,
1

I have tried different ways to solve my problems by looking various similar problems. I am getting Invalid argument error. When I tried to change sparse_categorical_crossentropy loss to categorical_crossentropy. So I am at a still at this still. Can any one help me guide through

InvalidArgumentError Traceback (most recent call last)
in
3 model = create_model(NUM_WORDS, EMBEDDING_DIM, MAXLEN)
4
----> 5 history = model.fit(train_padded_seq, train_label_seq, epochs=30, validation_data=(val_padded_seq, val_label_seq))

/opt/conda/lib/python3.8/site-packages/keras/utils/traceback_utils.py in error_handler(*args, **kwargs)
65 except Exception as e: # pylint: disable=broad-except
66 filtered_tb = _process_traceback_frames(e.traceback)
—> 67 raise e.with_traceback(filtered_tb) from None
68 finally:
69 del filtered_tb

/opt/conda/lib/python3.8/site-packages/tensorflow/python/eager/execute.py in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
56 try:
57 ctx.ensure_initialized()
—> 58 tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
59 inputs, attrs, num_outputs)
60 except core._NotOkStatusException as e:

InvalidArgumentError: indices[20,105] = 1372 is not in [0, 1000)
[[node sequential_5/embedding_5/embedding_lookup
(defined at /opt/conda/lib/python3.8/site-packages/keras/layers/embeddings.py:191)
]] [Op:__inference_train_function_3219]

Errors may have originated from an input operation.
Input Source operations connected to node sequential_5/embedding_5/embedding_lookup:
In[0] sequential_5/embedding_5/embedding_lookup/2990:
In[1] sequential_5/embedding_5/Cast (defined at /opt/conda/lib/python3.8/site-packages/keras/layers/embeddings.py:190)

Operation defined at: (most recent call last)

File “/opt/conda/lib/python3.8/runpy.py”, line 194, in _run_module_as_main
return _run_code(code, main_globals, None,

File “/opt/conda/lib/python3.8/runpy.py”, line 87, in _run_code
exec(code, run_globals)

File “/opt/conda/lib/python3.8/site-packages/ipykernel_launcher.py”, line 16, in
app.launch_new_instance()

File “/opt/conda/lib/python3.8/site-packages/traitlets/config/application.py”, line 845, in launch_instance
app.start()

File “/opt/conda/lib/python3.8/site-packages/ipykernel/kernelapp.py”, line 612, in start
self.io_loop.start()

File “/opt/conda/lib/python3.8/site-packages/tornado/platform/asyncio.py”, line 199, in start
self.asyncio_loop.run_forever()

File “/opt/conda/lib/python3.8/asyncio/base_events.py”, line 570, in run_forever
self._run_once()

File “/opt/conda/lib/python3.8/asyncio/base_events.py”, line 1859, in _run_once
handle._run()

File “/opt/conda/lib/python3.8/asyncio/events.py”, line 81, in _run
self._context.run(self._callback, *self._args)

File “/opt/conda/lib/python3.8/site-packages/tornado/ioloop.py”, line 688, in
lambda f: self._run_callback(functools.partial(callback, future))

File “/opt/conda/lib/python3.8/site-packages/tornado/ioloop.py”, line 741, in _run_callback
ret = callback()

File “/opt/conda/lib/python3.8/site-packages/tornado/gen.py”, line 814, in inner
self.ctx_run(self.run)

File “/opt/conda/lib/python3.8/site-packages/tornado/gen.py”, line 775, in run
yielded = self.gen.send(value)

File “/opt/conda/lib/python3.8/site-packages/ipykernel/kernelbase.py”, line 358, in process_one
yield gen.maybe_future(dispatch(*args))

File “/opt/conda/lib/python3.8/site-packages/tornado/gen.py”, line 234, in wrapper
yielded = ctx_run(next, result)

File “/opt/conda/lib/python3.8/site-packages/ipykernel/kernelbase.py”, line 261, in dispatch_shell
yield gen.maybe_future(handler(stream, idents, msg))

File “/opt/conda/lib/python3.8/site-packages/tornado/gen.py”, line 234, in wrapper
yielded = ctx_run(next, result)

File “/opt/conda/lib/python3.8/site-packages/ipykernel/kernelbase.py”, line 536, in execute_request
self.do_execute(

File “/opt/conda/lib/python3.8/site-packages/tornado/gen.py”, line 234, in wrapper
yielded = ctx_run(next, result)

File “/opt/conda/lib/python3.8/site-packages/ipykernel/ipkernel.py”, line 302, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)

File “/opt/conda/lib/python3.8/site-packages/ipykernel/zmqshell.py”, line 539, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)

File “/opt/conda/lib/python3.8/site-packages/IPython/core/interactiveshell.py”, line 2894, in run_cell
result = self._run_cell(

File “/opt/conda/lib/python3.8/site-packages/IPython/core/interactiveshell.py”, line 2940, in _run_cell
return runner(coro)

File “/opt/conda/lib/python3.8/site-packages/IPython/core/async_helpers.py”, line 68, in _pseudo_sync_runner
coro.send(None)

File “/opt/conda/lib/python3.8/site-packages/IPython/core/interactiveshell.py”, line 3165, in run_cell_async
has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

thank you
DP

2

I am not the mentor for that course. However, it is important to verify that the shape of your training labels is consistent with the loss function you are using.

3

according to course video it shows to use categorical crossentropy loss. but when I searched this issue here, somewhere it was stated to use sparse and yet my problem didn’t solve.

My problem looks like in embedding layer I suppose but I am unclear

Thank you
DP

4

I am not getting the desired accuracy

Epoch 1/30
56/56 [==============================] - 1s 4ms/step - loss: 1.5970 - accuracy: 0.2983 - val_loss: 1.5761 - val_accuracy: 0.4899
Epoch 2/30
56/56 [==============================] - 0s 2ms/step - loss: 1.5353 - accuracy: 0.5124 - val_loss: 1.4778 - val_accuracy: 0.5438
Epoch 3/30
56/56 [==============================] - 0s 2ms/step - loss: 1.3810 - accuracy: 0.6101 - val_loss: 1.2809 - val_accuracy: 0.6427
Epoch 4/30
56/56 [==============================] - 0s 2ms/step - loss: 1.1328 - accuracy: 0.7112 - val_loss: 1.0270 - val_accuracy: 0.7753
Epoch 5/30
56/56 [==============================] - 0s 2ms/step - loss: 0.8584 - accuracy: 0.8534 - val_loss: 0.7840 - val_accuracy: 0.8607
Epoch 6/30
56/56 [==============================] - 0s 2ms/step - loss: 0.6171 - accuracy: 0.9202 - val_loss: 0.5910 - val_accuracy: 0.9011
Epoch 7/30
56/56 [==============================] - 0s 2ms/step - loss: 0.4411 - accuracy: 0.9500 - val_loss: 0.4603 - val_accuracy: 0.9079
Epoch 8/30
56/56 [==============================] - 0s 2ms/step - loss: 0.3256 - accuracy: 0.9629 - val_loss: 0.3781 - val_accuracy: 0.9191
Epoch 9/30
56/56 [==============================] - 0s 2ms/step - loss: 0.2507 - accuracy: 0.9702 - val_loss: 0.3255 - val_accuracy: 0.9281
Epoch 10/30
56/56 [==============================] - 0s 2ms/step - loss: 0.1984 - accuracy: 0.9792 - val_loss: 0.2868 - val_accuracy: 0.9258
Epoch 11/30
56/56 [==============================] - 0s 3ms/step - loss: 0.1609 - accuracy: 0.9815 - val_loss: 0.2605 - val_accuracy: 0.9326
Epoch 12/30
56/56 [==============================] - 0s 2ms/step - loss: 0.1327 - accuracy: 0.9843 - val_loss: 0.2418 - val_accuracy: 0.9303
Epoch 13/30
56/56 [==============================] - 0s 2ms/step - loss: 0.1113 - accuracy: 0.9860 - val_loss: 0.2285 - val_accuracy: 0.9326
Epoch 14/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0939 - accuracy: 0.9904 - val_loss: 0.2170 - val_accuracy: 0.9303
Epoch 15/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0797 - accuracy: 0.9927 - val_loss: 0.2080 - val_accuracy: 0.9371
Epoch 16/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0677 - accuracy: 0.9933 - val_loss: 0.1998 - val_accuracy: 0.9371
Epoch 17/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0581 - accuracy: 0.9966 - val_loss: 0.1943 - val_accuracy: 0.9371
Epoch 18/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0501 - accuracy: 0.9972 - val_loss: 0.1903 - val_accuracy: 0.9393
Epoch 19/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0435 - accuracy: 0.9972 - val_loss: 0.1866 - val_accuracy: 0.9393
Epoch 20/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0376 - accuracy: 0.9978 - val_loss: 0.1829 - val_accuracy: 0.9438
Epoch 21/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0329 - accuracy: 0.9989 - val_loss: 0.1824 - val_accuracy: 0.9393
Epoch 22/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0288 - accuracy: 1.0000 - val_loss: 0.1796 - val_accuracy: 0.9416
Epoch 23/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0253 - accuracy: 0.9994 - val_loss: 0.1793 - val_accuracy: 0.9416
Epoch 24/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0224 - accuracy: 1.0000 - val_loss: 0.1773 - val_accuracy: 0.9393
Epoch 25/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0199 - accuracy: 1.0000 - val_loss: 0.1771 - val_accuracy: 0.9416
Epoch 26/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0177 - accuracy: 1.0000 - val_loss: 0.1762 - val_accuracy: 0.9416
Epoch 27/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0159 - accuracy: 1.0000 - val_loss: 0.1763 - val_accuracy: 0.9393
Epoch 28/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0143 - accuracy: 1.0000 - val_loss: 0.1755 - val_accuracy: 0.9393
Epoch 29/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0129 - accuracy: 1.0000 - val_loss: 0.1762 - val_accuracy: 0.9416
Epoch 30/30
56/56 [==============================] - 0s 2ms/step - loss: 0.0117 - accuracy: 1.0000 - val_loss: 0.1751 - val_accuracy: 0.9416

5

Sorry, I don’t have access to that course.
Hopefully, one of the mentors for this course will respond to your thread.

6

ok sir, no issue. thank you for responding.

7

For a multi-class classification problem, you can use categorical cross entropy when the true labels are one-hot encoded. If the true labels are not one-hot encoded, use the sparse version of the loss function.

Not getting the desired accuracy means that you’ll have to change the model architecture. We’ve had a similar discussion in the recent past. Here’s the topic. Do see this link on underfitting and overfitting

If that doesn’t help, please click my name and message your notebook as an attachment.

1 Like
8

:grimacing: Heee heee yes we had similar discussion, but I think here the problem might be with embedding layer. I am not sure :). I have shared the notebook.

Thank you
DP

9

what do you mean by true labels are one-hot encoded?

10

Few things to keep in mind:

  1. Not all points shared in the other topic have been incorporated. Here are a few:
    a. Number of units of hidden layers is not a power of 2.
    b. Model complexity can be increased when underfitting happens. Since the embedding layer configuration is specified by the staff, I recommend you play with layers other than the Embedding layer and the final output layer.
  2. Do not hardcode values for the Embedding layer. Please rely on constants defined at the top of the notebook like EMBEDDING_DIM. Do follow this fix across the notebook where applicable. Rely on function parameters and then constants when defined.

True labels are the actual labels that are part of the dataset you want the model to train on. Labels generated by the model are called as predictions. When all predicted labels match the actual labels, we have 100% accuracy. Please read the section on Machine learning and statistics here to learn about one-hot encoding.

Please complete the deep learning specialization before this one to get a good understanding of model analysis.

1 Like
11

adding number of units of hidden layers with a power of 2 created the invalid argument error.

I changed the embedding layer specified according to the assignment that also caused the invalid argument error.

please check your inbox once.

12

input_dim of the Embedding layer should correspond to the number of words in the vocabulary. Please look at the function argument to figure this out instead of making it 1000 manually.

13

yes right I used the number of words in vocabulary and I got the accuracy but then in the next section by output was not matching with expected out.

14

What do you mean by?

1 Like
15

if I changed to num_words to number of words in of vocabulary I got this issue, so I went back to changing num_words to 1000 that is when I keep getting all these errors, so getting confused.

image
image1117×405 50.3 KB

16

Vocabulary size is determined by the tokenizer that encodes words as integers. For this assignment, don’t set this value to 1000 but rely on the correct function parameter to create_model.

In your personal projects, setting num_words parameter of Tokenizer is the right way to restrict the vocabulary.

17

if I do not use num word as 1000 then my next section after the model training does not match with the expected output.

19

Balaji I used num_word 1000 only and I cleared the model.

at the beginning of the assignment it is mentioned to use num words 1000. I was doing a mistake in noticing one. I needed to have training accuracy of 95 and validation 90% which I was getting. but I thought like previous week accuracy I needed to get 99% training accuracy and 95% accuracy for validation.

I cleared the assignment grade too.

20

Congrats. As mentioned before, don’t hardcode values but rely on the function parameter and tokenizer state. It’s possible for the grader to use a different configuration than the one shared in the starter code.

21

I am having a similar error. I have checked my code over and over again and I cant find what’s wrong. It now stops processing when I reach this cell

grader-required-cell

Test your function

train_label_seq = tokenize_labels(labels, train_labels)
val_label_seq = tokenize_labels(labels, val_labels)

I have ran it with the same code and have not received the error message shared in the original post , and it hasn’t stopped running. No issues with connectivity my GPU or any other factors I have deleted and restarted the notebook multiple times. If you could not respond by commenting this is due to a lack of knowledge I should have when starting this paper, I am on the third stage of the accreditation, I am well versed in python and ruby and have completed a previous tensorflow keras course including NLP.