# GRADED FUNCTION EncoderLayer

**URL:** <https://community.deeplearning.ai/t/graded-function-encoderlayer/100260>\
**Category:** Sequence Models\
**Tags:** coursera-platform\
**Created:** [February 17, 2022, 2:08pm UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260 "2022-02-17T14:08:33Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![RogerH](https://avatars.discourse-cdn.com/v4/letter/r/7ba0ec/32.png) [@RogerH](https://community.deeplearning.ai/u/RogerH)\
**Post date:** [February 17, 2022, 2:08pm UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/1 "2022-02-17T14:08:34Z")

</div>

Pretty sure my code is good, but get this error

ameError Traceback (most recent call last)  
 in   
1 # UNIT TEST  
----\> 2 EncoderLayer\_test(EncoderLayer)

~/work/W4A1/public\_tests.py in EncoderLayer\_test(target)  
82 def EncoderLayer\_test(target):  
83 q = np.array([[[1, 0, 1, 1], [0, 1, 1, 1], [1, 0, 0, 1]]]).astype(np.float32)  
—\> 84 encoder\_layer1 = target(4, 2, 8)  
85 tf.random.set\_seed(10)  
86 encoded = encoder\_layer1(q, True, np.array([[1, 0, 1]]))

in **init** (self, embedding\_dim, num\_heads, fully\_connected\_dim, dropout\_rate, layernorm\_eps)  
16 dropout=dropout\_rate)  
17  
—\> 18 self.ffn = FullyConnected(embedding\_dim=embedding\_dim,  
19 fully\_connected\_dim=fully\_connected\_dim)  
20

NameError: name ‘FullyConnected’ is not defined

---

<div class="post-metadata">

**Author:** ![TMosh](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/tmosh/32/9321_2.png) [@TMosh](https://community.deeplearning.ai/u/TMosh)\
**Post date:** [February 17, 2022, 5:03pm UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/2 "2022-02-17T17:03:26Z")

</div>

Try restarting the kernel and running all the cells again.

---

<div class="post-metadata">

**Author:** ![RogerH](https://avatars.discourse-cdn.com/v4/letter/r/7ba0ec/32.png) [@RogerH](https://community.deeplearning.ai/u/RogerH)\
**Post date:** [February 19, 2022, 2:38pm UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/3 "2022-02-19T14:38:56Z")

</div>

Worked like a charm, thank you

---

<div class="post-metadata">

**Author:** ![Darshan\_K1](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@Darshan\_K1](https://community.deeplearning.ai/u/Darshan_K1)\
**Post date:** [July 13, 2023, 10:48am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/4 "2023-07-13T10:48:38Z")

</div>

Hello, i have the same problem. I already try to restart the kernel. Do you have any idea what is going on?

---

<div class="post-metadata">

**Author:** ![saifkhanengr](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/saifkhanengr/32/152858_2.png) [@saifkhanengr](https://community.deeplearning.ai/u/saifkhanengr)\
**Post date:** [July 13, 2023, 10:59am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/5 "2023-07-13T10:59:51Z")

</div>

Restarting the Kernel alone is not enough. You have to run all the cells that come before that particular cell (EncoderLayer cell).

---

<div class="post-metadata">

**Author:** ![Darshan\_K1](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@Darshan\_K1](https://community.deeplearning.ai/u/Darshan_K1)\
**Post date:** [July 13, 2023, 11:02am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/6 "2023-07-13T11:02:10Z")

</div>

Thanks for the answer, I already did that.

---

<div class="post-metadata">

**Author:** ![saifkhanengr](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/saifkhanengr/32/152858_2.png) [@saifkhanengr](https://community.deeplearning.ai/u/saifkhanengr)\
**Post date:** [July 13, 2023, 11:04am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/7 "2023-07-13T11:04:22Z")

</div>

Please answer these:

1. Have you deleted any cell(s)?
2. Have you changed the code of ungraded cell(s) or at any other place which you were not supposed to do?
3. Have you passed all the above tests?

---

<div class="post-metadata">

**Author:** ![Darshan\_K1](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@Darshan\_K1](https://community.deeplearning.ai/u/Darshan_K1)\
**Post date:** [July 13, 2023, 11:06am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/8 "2023-07-13T11:06:37Z")

</div>

1. & 2. I did nothing of these but maybe in a moment of distraction. If I have to answer I will say no.
2. Yes, I passed all above tests. I can sh

---

<div class="post-metadata">

**Author:** ![Darshan\_K1](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@Darshan\_K1](https://community.deeplearning.ai/u/Darshan_K1)\
**Post date:** [July 13, 2023, 11:07am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/9 "2023-07-13T11:07:39Z")

</div>

Here is the EncoderLayer cell without my answers:

# UNQ\_C4 (UNIQUE CELL IDENTIFIER, DO NOT EDIT)

# GRADED FUNCTION EncoderLayer

class EncoderLayer(tf.keras.layers.Layer):  
“”"  
The encoder layer is composed by a multi-head self-attention mechanism,  
followed by a simple, positionwise fully connected feed-forward network.  
This architecture includes a residual connection around each of the two  
sub-layers, followed by layer normalization.  
“”"  
def **init** (self, embedding\_dim, num\_heads, fully\_connected\_dim,  
dropout\_rate=0.1, layernorm\_eps=1e-6):  
super(EncoderLayer, self). **init** ()

```
    self.mha = MultiHeadAttention(num_heads=num_heads,
                                  key_dim=embedding_dim,
                                  dropout=dropout_rate)

    self.ffn = FullyConnected(embedding_dim=embedding_dim,
                              fully_connected_dim=fully_connected_dim)

    self.layernorm1 = LayerNormalization(epsilon=layernorm_eps)
    self.layernorm2 = LayerNormalization(epsilon=layernorm_eps)

    self.dropout_ffn = Dropout(dropout_rate)

def call(self, x, training, mask):
    """
    Forward pass for the Encoder Layer
    
    Arguments:
        x -- Tensor of shape (batch_size, input_seq_len, fully_connected_dim)
        training -- Boolean, set to true to activate
                    the training mode for dropout layers
        mask -- Boolean mask to ensure that the padding is not 
                treated as part of the input
    Returns:
        encoder_layer_out -- Tensor of shape (batch_size, input_seq_len, embedding_dim)
    """

```

---

<div class="post-metadata">

**Author:** ![Darshan\_K1](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@Darshan\_K1](https://community.deeplearning.ai/u/Darshan_K1)\
**Post date:** [July 13, 2023, 11:10am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/10 "2023-07-13T11:10:24Z")

</div>

I will try to delete the .ipynb file and start a new one.

---

<div class="post-metadata">

**Author:** ![saifkhanengr](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/saifkhanengr/32/152858_2.png) [@saifkhanengr](https://community.deeplearning.ai/u/saifkhanengr)\
**Post date:** [July 13, 2023, 11:11am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/11 "2023-07-13T11:11:53Z")

</div>

Please let us know if getting a fresh copy of your assignment resolve the issue.

---

<div class="post-metadata">

**Author:** ![Darshan\_K1](https://avatars.discourse-cdn.com/v4/letter/d/e5b9ba/32.png) [@Darshan\_K1](https://community.deeplearning.ai/u/Darshan_K1)\
**Post date:** [July 13, 2023, 11:19am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/12 "2023-07-13T11:19:10Z")

</div>

I’m back.  
I notice this time there is this cell which defines the layer. I haven’t seen it before, maybe I deleted it.  
Thanks for your help.

def FullyConnected(embedding\_dim, fully\_connected\_dim):  
return tf.keras.Sequential([  
tf.keras.layers.Dense(fully\_connected\_dim, activation=‘relu’), # (batch\_size, seq\_len, dff)  
tf.keras.layers.Dense(embedding\_dim) # (batch\_size, seq\_len, d\_model)  
])

---

<div class="post-metadata">

**Author:** ![saifkhanengr](https://sea1.discourse-cdn.com/dlai/user_avatar/community.deeplearning.ai/saifkhanengr/32/152858_2.png) [@saifkhanengr](https://community.deeplearning.ai/u/saifkhanengr)\
**Post date:** [July 13, 2023, 11:27am UTC](https://community.deeplearning.ai/t/graded-function-encoderlayer/100260/13 "2023-07-13T11:27:30Z")

</div>

👍 👍 👍🙂 🙂 🙂 ~~~~~~~~~~
