not able to understand why * So, the shape of a mini-batch is (ππ₯,π,ππ₯)* this shape is used in RNN.
First of all, please clarify where are you referring ? Video ? Assignment of which week ?
Assuming that you refer W2A1 assignment, here is the way to think about the dimension.
Of course, if you design the system, you can chose your preferable definition.
In this assessment, we handle the time series data. Its length is Tx. And, data x is one dimensional input vector. The length of the vector is n_x. We want to handle multiple data at one time. The size of this mini-batch is m. So, the input data is, of course, 3-dimensional array.
Then, the design is which should be which dimension.
As this is the time-series data, we want to process data at time t at one time. In this sense, it is very natural to set βtimeβ to the last dimension like this.
When we get a slice at tiime T_n, it will be 2D matrix, with the dimension reduction for the last axis. I think all agree this point.
Then, the next design point is which (n_x or m) should be the first dimension ? Actually, this is up-to your design. But, one design point is how we can perform all calculations efficiently.
In RNN cell, there are several matrix operation (dot product). If we see rnn_cell_forward(), a time slice of input data, x_t is used in here. Itβs np.dot(W_{ax},\ x_t).
From a dot product view point, it is better for x_t to have one sample in a column, not row. (Otherwise, we need to βTransposeβ it.).
So, the shape of each time slice is as follows.
In net, the input shape becomes (n_x, m, T_x).
Of course, this dimension definition is for this assignment. You can design any input shape with considering data format, slices, definitions of other variables (Shape, etc.), and performance.
2 Likes
I am confuse what (nx, m) represent, is it a complete sentence of m words having nx length. if that is the case, then Tx was supposed to represent the word at T1, T2 β¦ Tx right?
Itβs really simple. Letβs say you have 3 words. βThe cat satβ. In above image m1 = βTheβ, m2 = βcatβ, m3 = βsatβ. Now each mβs have nx number of rows. In case of week 1 assignment 1. Letβs say xt_tmp = (3,3). Then, βTheβ has 3 rows. βcatβ also has 3 rows and βsatβ also has 3 rows (see the image above).