C2_W1_A1: three-layer neural network description

Course Q&A Deep Learning Specialization Improving Deep Neural Networks: Hyperparameter tun
1

Greetings!

How do I read “LINEAR->RELU->LINEAR->RELU->LINEAR->SIGMOID” correctly to understand the structure of the three-layer neural network?

I do understand LINEAR and RELU, but a three-layer neural network should look like “RELU->RELU->SIGMOID”…

Thank you.

image
image1447×684 50.9 KB

2

For this assignment, look at def forward_propagation(X, parameters): inside init_utils.py that does this for you.

In the general sense, Dense layer has activation parameter that applies a transformation function after performing affine operation (WX + b).

3

The notation is a little bit misleading. I would have written that as:

[Linear with ReLU activation]->
[Linear with ReLU activation]->
[Linear with sigmoid activation]

4

What does Linear mean here?
Is this transformation of previous-layer activations before passing to the current-layer activation function?

5

OK, I see it now.
A bit forensic investigation always helps :grinning_face:

So the “LINEAR->RELU” layer first does affine operation (WX + b) and then applies ReLu activation function.

In this case I would suggest to slightly change the notation to “(LINEAR->RELU)->(LINEAR->RELU)->(LINEAR->SIGMOID)” or “(LINEAR | RELU) → (LINEAR | RELU) → (LINEAR | SIGMOID)”.

Does it make sense?

6

Correct :100:

Yes…

7

Thanks for your suggestion.