(and sorry, I wish I had a better subject, but I’m not quite sure how to qualify my questions here)
The first one is really, more or less straight forward:
In Tensorflow, how should we think about the input_dim and output_dim of an embedding layer ?
Usually I can see the input_dim, but I’m often confused where we are getting the output_dim from.
The second question is a little bit more complicated, and more a Python issue. Especially at the tail end of NLP they kind of shift into ‘high gear’ with the Python OOP model. I’ve worked with objects before, and realize this is not a Python class, but I’ve been struggling a lot more with thinking about how we complete the object, or reference items within itself.
i.e. for example, when to use ‘self’. Also, here they have an def __init__ but then a def call. If anything I thought it was supposed to be def __call__. Otherwise what is calling ‘call’?
I’m just getting a little more lost here.
(There is also this line… I have nooo idea what is going on here: super(Encoder, self).__init__() because it is contained already within the original __init__.
input_dim is the size of the vocabulary.
output_dim is the size of the embedding vector (how many features each word will have).
self is the instance of the class. It’s used to access attributes and methods within the class. e.g., in self.attribute, self tells Python that you’re referring to the instance’s attribute.
__init__(self) is the constructor method, which runs when an instance of the class is created. It initializes the object’s state (i.e., setting initial values for attributes).
super(Encoder, self).__init__() is used to call the parent class’s __init__ method.
In TensorFlow, most models have both __init__ and call. The call method typically defines the forward pass logic, and TensorFlow internally handles invoking it. call in TensorFlow is conceptually similar to forward in PyTorch. On the other hand, __call__(self) is a special method that allows an instance of the class to be called as if it were a function. A different thing entirely.
2 Likes
@lukmanaj thank you, especially the difference between def call and __call__ and super(Encoder, self).__init__(). Those were the parts I was most confused.
I mean I understood what __call__ would do but that wasn’t happening here. I didn’t know this was a Tensorflow thing. Unfortunately I haven’t taken one of the Tensorflow specializations yet, so have just been trying to learn while following along with DLS/NLP.
Let me give another shot at the assignment tomorrow and get back to you if I still have questions. Thnx.
You are welcome @Nevermnd
@lukmanaj Great ! That was so helpful and I understand so much better now and it is clear to me now what is going on. I was able to now complete exercise 1 in about a minute provided your advice. Now, on to the rest of the assignment.
1 Like