Hi everyone,

While implementing the Dense Layer class, I produced the expected weights and forward function output in the first test.

However, in my second test, it failed to return the forward output with the following error message:

TypeError: dot_general requires contracting dimensions to have the same shape, got [4] and [3].

In the unittest, I got the same error message.

I suspect it has something to do with dense or w, but cannot identify the solution. Help appreciated!

Hi @Artie_Lam

Iâ€™m guessing the most probable place for your mistake is weights initialization in `init_weights_and_state`

function.

You have two lines of code in this function. The first one is pretty easy:

`input_shape = None`

Here you just have to assign the value of `input_signature`

attribute `.shape`

.

Second line is a bit trickier:

` # Generate the weight matrix from a normal distribution,`

` # and standard deviation of 'stdev' `

` w = None * trax.fastmath.random.normal(key = None, shape = (None, None))`

The first `None`

is the value of standard deviation for initializing weights that has default value of 0.1 and is assigned when initializing this class in `self._init_stdev`

.

The second `None`

is random key that `fastmath.random.normal`

function requires. Naturally this would be the argument `random_key`

.

The third and fourth `None`

s is where you probably made a mistake. The shape of the weight matrix for Dense layer should be (*â€śthe last dimension of inputsâ€ť x â€śthe last dimension of outputsâ€ť*). For example, if your X (inputs) for Dense layer is of shape *(4, 5)* and you want your output to be the shape of *(4, 10)*, then the w should have a shape of *(5, 10)*, because: *(4, 5) dot (5, 10)* â†’ *(4, 10)*.

So this is all to explain that the `shape=`

should be (*â€śthe last dimension of *`input_shape`

â€ť x *â€śn_units of the output dimensionâ€ť*). You can get the last dimension with `[-1]`

and `n_units`

is saved in the class attribute `self._n_units`

.

Cheers

2 Likes