Week 1 Assignment 1 - What is a RNN Cell?

Assignment 1 says:

You can think of the recurrent neural network as the repeated use of a single cell. First, you’ll implement the computations for a single time step.

  • What does this mean in terms of neurons?
  • Is RNN cell equivalent to a single neuron?
    I really want to connect it with we studied in Course 1 in terms of feed-forward networks.

Hi there @deepakjangra

A single RNN cell is not equivalent to a single neuron, but rather a small neural network in itself that contains multiple neurons. This structure is different from feed-forward networks, which only pass information forward through the layers without any memory of past inputs.

Imagine a single RNN cell as a feed-forward network layer with recurrent connections. It processes an input, updates its hidden state, and produces an output. This process is repeated at each time step, with the hidden state being passed along, allowing the network to remember and use past information.

Let’s compare Feedforward NNs and RNNs:

  • In a Feedforward NN, you have layers of neurons where each neuron computes a weighted sum of inputs, applies an activation function, and passes the output to the next layer.

  • An RNN cell contains multiple neurons arranged in such a way that they can maintain and update a hidden state and it has recurrent connections. This means that the output of the RNN cell at one time step is fed back into the cell as part of the input for the next time step.

Hope this helps, feel free to ask if you need further assistance!