not able to understand why * So, the shape of a mini-batch is (๐๐ฅ,๐,๐๐ฅ)* this shape is used in RNN.
First of all, please clarify where are you referring ? Video ? Assignment of which week ?
Assuming that you refer W2A1 assignment, here is the way to think about the dimension.
Of course, if you design the system, you can chose your preferable definition.
In this assessment, we handle the time series data. Its length is Tx. And, data x is one dimensional input vector. The length of the vector is n_x. We want to handle multiple data at one time. The size of this mini-batch is m. So, the input data is, of course, 3-dimensional array.
Then, the design is which should be which dimension.
As this is the time-series data, we want to process data at time t at one time. In this sense, it is very natural to set โtimeโ to the last dimension like this.
When we get a slice at tiime T_n, it will be 2D matrix, with the dimension reduction for the last axis. I think all agree this point.
Then, the next design point is which (n_x or m) should be the first dimension ? Actually, this is up-to your design. But, one design point is how we can perform all calculations efficiently.
In RNN cell, there are several matrix operation (dot product). If we see rnn_cell_forward(), a time slice of input data, x_t is used in here. Itโs np.dot(W_{ax},\ x_t).
From a dot product view point, it is better for x_t to have one sample in a column, not row. (Otherwise, we need to โTransposeโ it.).
So, the shape of each time slice is as follows.
In net, the input shape becomes (n_x, m, T_x).
Of course, this dimension definition is for this assignment. You can design any input shape with considering data format, slices, definitions of other variables (Shape, etc.), and performance.
I am confuse what (nx, m) represent, is it a complete sentence of m words having nx length. if that is the case, then Tx was supposed to represent the word at T1, T2 โฆ Tx right?
Itโs really simple. Letโs say you have 3 words. โThe cat satโ. In above image m1 = โTheโ, m2 = โcatโ, m3 = โsatโ. Now each mโs have nx number of rows. In case of week 1 assignment 1. Letโs say xt_tmp = (3,3). Then, โTheโ has 3 rows. โcatโ also has 3 rows and โsatโ also has 3 rows (see the image above).