# Transformer Error **URL:** \ **Category:** Sequence Models\ **Tags:** coursera-platform\ **Created:** [August 3, 2021, 3:08pm UTC](https://community.deeplearning.ai/t/transformer-error/24906 "2021-08-03T15:08:23Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![jose.bulatao](https://avatars.discourse-cdn.com/v4/letter/j/779978/32.png) [@jose.bulatao](https://community.deeplearning.ai/u/jose.bulatao)\ **Post date:** [August 3, 2021, 3:08pm UTC](https://community.deeplearning.ai/t/transformer-error/24906/1 "2021-08-03T15:08:23Z")
I’m running into an error on Week 4’s Assignment of Course 5. Any tips are greatly appreciated. Thanks in advance. JB ERROR: ## positions 5 d 4 angle\_rads [[0. 0. 0. 0.] [1. 1. 0.01 0.01] [2. 2. 0.02 0.02] [3. 3. 0.03 0.03] [4. 4. 0.04 0.04]] angle rad shape (5, 4) positions 6 d 4 angle\_rads [[0. 0. 0. 0.] [1. 1. 0.01 0.01] [2. 2. 0.02 0.02] [3. 3. 0.03 0.03] [4. 4. 0.04 0.04] [5. 5. 0.05 0.05]] angle rad shape (6, 4) InvalidArgumentError Traceback (most recent call last) in 1 # UNIT TEST ----\> 2 Transformer\_test(Transformer, create\_look\_ahead\_mask, create\_padding\_mask) ~/work/W4A1/public\_tests.py in Transformer\_test(target, create\_look\_ahead\_mask, create\_padding\_mask) 276 enc\_padding\_mask, 277 look\_ahead\_mask, → 278 dec\_padding\_mask 279 ) 280 /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base\_layer.py in **call** (self, \*args, \*\*kwargs) 1010 with autocast\_variable.enable\_auto\_cast\_variables( 1011 self.\_compute\_dtype\_object): → 1012 outputs = call\_fn(inputs, \*args, \*\*kwargs) 1013 1014 if self.\_activity\_regularizer: in call(self, input\_sentence, output\_sentence, training, enc\_padding\_mask, look\_ahead\_mask, dec\_padding\_mask) 56 # call self.decoder with the appropriate arguments to get the decoder output 57 # dec\_output.shape == (batch\_size, tar\_seq\_len, fully\_connected\_dim) —\> 58 dec\_output, attention\_weights = self.decoder(enc\_output, output\_sentence, training, look\_ahead\_mask, dec\_padding\_mask) 59 60 # pass decoder output through a linear layer and softmax (~2 lines) /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base\_layer.py in **call** (self, \*args, \*\*kwargs) 1010 with autocast\_variable.enable\_auto\_cast\_variables( 1011 self.\_compute\_dtype\_object): → 1012 outputs = call\_fn(inputs, \*args, \*\*kwargs) 1013 1014 if self.\_activity\_regularizer: in call(self, x, enc\_output, training, look\_ahead\_mask, padding\_mask) 49 # START CODE HERE 50 # create word embeddings —\> 51 x = self.embedding(x) # (batch\_size, target\_seq\_len, fully\_connected\_dim) 52 53 # scale embeddings by multiplying by the square root of their dimension /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/engine/base\_layer.py in **call** (self, \*args, \*\*kwargs) 1010 with autocast\_variable.enable\_auto\_cast\_variables( 1011 self.\_compute\_dtype\_object): → 1012 outputs = call\_fn(inputs, \*args, \*\*kwargs) 1013 1014 if self.\_activity\_regularizer: /opt/conda/lib/python3.7/site-packages/tensorflow/python/keras/layers/embeddings.py in call(self, inputs) 194 out = embedding\_ops.embedding\_lookup\_v2(self.embeddings.variables, inputs) 195 else: → 196 out = embedding\_ops.embedding\_lookup\_v2(self.embeddings, inputs) 197 if self.\_dtype\_policy.compute\_dtype != self.\_dtype\_policy.variable\_dtype: 198 # Instead of casting the variable as in most layers, cast the output, as /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(\*args, \*\*kwargs) 199 “”“Call target, and fall back on dispatchers if there is a TypeError.”“” 200 try: → 201 return target(\*args, \*\*kwargs) 202 except (TypeError, ValueError): 203 # Note: convert\_to\_eager\_tensor currently raises a ValueError, not a /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/embedding\_ops.py in embedding\_lookup\_v2(params, ids, max\_norm, name) 392 ValueError: If `params` is empty. 393 “”" → 394 return embedding\_lookup(params, ids, “div”, name, max\_norm=max\_norm) 395 396 /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(\*args, \*\*kwargs) 199 “”“Call target, and fall back on dispatchers if there is a TypeError.”“” 200 try: → 201 return target(\*args, \*\*kwargs) 202 except (TypeError, ValueError): 203 # Note: convert\_to\_eager\_tensor currently raises a ValueError, not a /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/embedding\_ops.py in embedding\_lookup(params, ids, partition\_strategy, name, validate\_indices, max\_norm) 326 name=name, 327 max\_norm=max\_norm, → 328 transform\_fn=None) 329 330 /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/embedding\_ops.py in \_embedding\_lookup\_and\_transform(params, ids, partition\_strategy, name, max\_norm, transform\_fn) 136 with ops.colocate\_with(params[0]): 137 result = \_clip( → 138 array\_ops.gather(params[0], ids, name=name), ids, max\_norm) 139 if transform\_fn: 140 result = transform\_fn(result) /opt/conda/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py in wrapper(\*args, \*\*kwargs) 199 “”“Call target, and fall back on dispatchers if there is a TypeError.”“” 200 try: → 201 return target(\*args, \*\*kwargs) 202 except (TypeError, ValueError): 203 # Note: convert\_to\_eager\_tensor currently raises a ValueError, not a /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/array\_ops.py in gather(_ **failed resolving arguments** _) 4811 # TODO(apassos) find a less bad way of detecting resource variables 4812 # without introducing a circular dependency. → 4813 return params.sparse\_read(indices, name=name) 4814 except AttributeError: 4815 return gen\_array\_ops.gather\_v2(params, indices, axis, name=name) /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/resource\_variable\_ops.py in sparse\_read(self, indices, name) 701 variable\_accessed(self) 702 value = gen\_resource\_variable\_ops.resource\_gather( → 703 self.\_handle, indices, dtype=self.\_dtype, name=name) 704 705 if self.\_dtype == dtypes.variant: /opt/conda/lib/python3.7/site-packages/tensorflow/python/ops/gen\_resource\_variable\_ops.py in resource\_gather(resource, indices, dtype, batch\_dims, validate\_indices, name) 547 return \_result 548 except \_core.\_NotOkStatusException as e: → 549 \_ops.raise\_from\_not\_ok\_status(e, name) 550 except \_core.\_FallbackException: 551 pass /opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py in raise\_from\_not\_ok\_status(e, name) 6860 message = e.message + (" name: " + name if name is not None else “”) 6861 # pylint: disable=protected-access → 6862 six.raise\_from(core.\_status\_to\_exception(e.code, message), None) 6863 # pylint: enable=protected-access 6864 /opt/conda/lib/python3.7/site-packages/six.py in raise\_from(value, from\_value) InvalidArgumentError: indices[0,0,2] = -1 is not in [0, 35) [Op:ResourceGather] ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [August 3, 2021, 3:31pm UTC](https://community.deeplearning.ai/t/transformer-error/24906/2 "2021-08-03T15:31:11Z")
> [@jose.bulatao](#): > > dec\_output, attention\_weights = self.decoder(enc\_output, output\_sentence, training, look\_ahead\_mask, dec\_padding\_mask) The order of the first two arguments are reversed. ---
**Author:** ![jose.bulatao](https://avatars.discourse-cdn.com/v4/letter/j/779978/32.png) [@jose.bulatao](https://community.deeplearning.ai/u/jose.bulatao)\ **Post date:** [August 3, 2021, 3:54pm UTC](https://community.deeplearning.ai/t/transformer-error/24906/3 "2021-08-03T15:54:24Z")
Thanks Tom! I appreciate all your help throughout the course! JB ---
**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\ **Post date:** [July 17, 2023, 3:20pm UTC](https://community.deeplearning.ai/t/transformer-error/24906/4 "2023-07-17T15:20:36Z")
Closed on July 17, 2023, 3:20pm UTC