Dimensions of input x

It is a very basic question, but I am still confused, based on the follwing information from the assignment notebook.

"Input with 𝑛π‘₯ number of units

For a single time step of a single input example,


is a one-dimensional input vector

Using language as an example, a language with a 5000-word vocabulary could be one-hot encoded into a vector that has 5000 units. So


would have the shape (5000,)


So, in this case, 1 word will be represented as a vector shape (5000,) with only one β€œ1” and 4999 β€œ0” in that vector, right? So why is that a one-dimensional input vector (with 5000 units) instead of a 5000-dimensional vector? I don’t get the difference among units and dimensions, especially regarding language


The number of dimensions is the number of coordinates needed to identify each value.

For example, a 2D matrix would have two coordinates, one for the rows and one for the columns. It might be noted as X(r,c).

A 3D matrix would have three coordinates, for example a color image would have a rows and column for each of three colors (red, blue, green). It might be noted as A(x,y,3).

The number of elements in a dimension is a different concept than the number of dimensions.

Thank you for the answer. Still have some doubts. So, the coordinates to represent a word vector after one-hot encoding of shape (5000,) is only one? How that vector will be represented in the vectorial space? Different words would give vectors in the same axis with only different lengths? And what if the word vector has not only one β€œ1” but two β€œ1” in a vector shape (5000,)? Would that be also 1 dimensional?

All the vectors would be the same length, with 5000 elements. In each vector, 4999 elements are zeros, and there is a single 1.

I feel there is a misconception here between classic math and CS/NumPy/assignment notebook terminology.
In math, a vector of 5000 values is called a 5000-dimensional vector.
However, in numpy, a vector of shape (5000,) is a rank-1 ndarray, sometimes called 1-dimensional array / 1-dimensional vector (meaning a 1 sequence of numbers, in 1 dimension).

Values β€œinside” vectors do not affect dimensionality in both terminologies.

Thanks. That makes sense now.