How to determine the input format of tensor for the evaluation/prediction mode in Trax

Course Q&A Natural Language Processing NLP Resources
1

Hi all,

For C4_W2_Assignment in the Natural Language Processing Specialization, I found it kind of confusing for determining the input for the transformer model(See attached fig). I think originally for training the model, we need three arrays (X,Y and mask), while here the input only has a dimension of two??? We should we neglect the mask array?

More in general, is there a way to better understanding the input-dimension and output-dimension?

image
image728Ă—57 8.4 KB

2

This is complex model with many parts and you need to go through from the beginning in order to understand how those parts are put together to arrive at the explained output.

3

Hi @David_C1

There was a previous similar question that might suggest you some ideas.

Cheers

1 Like
4

Hi Gent, thanks for replying. I wonder is there a systematic way to understand the input/output parameters of each layer or the whole model in Trax? Thanks!

5

Hi ArvyZukai, thanks for replying. I wonder is there a systematic way to understand the input/output parameters of each layer or the whole model in Trax? Thanks!

6

Hi @David_C1

I’m not sure I can better explain than the trax documentation (especially “Layers are trainable.”, " Layers combine into layers." sections and “2. Inputs and Outputs”).

You can play around in your Coursera labs by trying to initialize simple tl.Dense() layer with your own input (don’t forget to initialize with input.signature).

Cheers

7

Hello @David_C1 you could follow up through the notebook and try understand as much as you can. Other external resources that Arvydas suggests can help also.