W4 Assignment 1 Exercise 8: Question about the custom Transformer model's call method

Course Q&A Deep Learning Specialization Sequence Models
1

The Model implemented in the assignment has the following “call” method

def call(self, x, enc_output, training, look_ahead_mask, padding_mask):

I figured this method is somehow invoked when model.fit() is called which I’m trying to figure out how to do and in the process it turns out the call method’s signature is wrong according to the documentation it must be:

def call(self, inputs, training=None, mask=None)

When I try to call model.fit() with some inputs (probably wrong shape) I get the following error:

Models passed to `fit` can only have `training` and the first argument in `call()` as positional arguments, found: ['output_sentance', 'enc_padding_mask', 'look_ahead_mask', 'dec_padding_mask'].

Is my assumption for the method’s signature correct? Also I would appreciate any help in my attempt to train the transformer model implemented in the notebook with the simplest dataset possible.

2

It appears you are asking specifically about the DecoderLayer() class definition.

Since this is a custom layer, it can use whatever call arguments are needed for this specific task.

The DecoderLayer() class is used from the Transformer() class (see Exercise 8), when it calls the decoder() method, which calls the Decoder() class.

Then the Decoder() class calls DecoderLayer() via its dec_layers() method.

This notebook does not actually do any training - all this assignment does is create a Transformer model.

Week 4 also includes some ungraded labs that show how transformer models are used.

3

Thanks for the clarification. You’re right. I was not correct to assume the call method signature is wrong since it’s being invoked only from code we’ve written. Still I’m curious if it’s possible this transformer model to be trained using model.fit() or a custom training loop is required for that?

4

Yes, it is.

It’s disappointing that the notebook doesn’t demonstrate this.