C3W2 Train accuracy high but validation accuracy very low

Course Q&A TensorFlow Developer Professional Natural Language Processing in TensorFlow
1

When creating a post, please add:

  • Week # must be added in the tags option of the post.
  • Link to the classroom item you are referring to: C3W2_Assignment.ipynb

Int the final C3W2 assignment, I got a model passing all the tests, but when I ran fit, the training accuracy gets to 0.99 but val_accuracy stays around 0.2. I tried many variations of the parameters in the model and different optimizers (‘adam’, ‘rmsprop’), but the results do not change much. Anyone has any suggestions? Or seen similar results?

2

this would mean your model dataset is underfitting if considering training accuracy achieving 0.99

did you look at the labels and text being correctly split and assigned to the dataset?

if I had this issue, I would start from my first grade function, check if I have correctly written the codes.

Also make sure your previous test cell have passed as expected.

3

Thank you for your quick reply. I checked each test code, they all produce the expected results (or passed). I’ll re-examine my code.

4

I’m confused why I can’t use
vectorized_text = text_vectorizer(text_only)
where text_only is splitted out from the data passed in preprocess_dataset(), instead I have to use
vectorized_text = text_only.map(text_vectorizer)
This first gives me an error, but I suspect the 2nd does not do what I need to do. Am I missing something here?

5

ok your msg is bit confusing, can you first tell me which one did you use and which grade function cell you used?

check this thread

Also make sure your model follows the architecture instructions give in the assignment, like using the number layers and then dense output layers.

Also confirm what loss you have used categorical crossentropy or sparse categorical crossentropy??

remember here classes are divided into labels, so sparse would be the choice right?

6

[quote=“Deepti_Prasad, post:5, topic:770525”]
I’m using sparse categorical crossentropy.
I was talking about within preprocess_dataset(), a dataset, a text_vectorizer and a label_encoder is passed in. First I extract text_only and label_only by using map(). Then I’m supposed to use the text_vectorizer in text_only and label_encoder on the label_only.

7

in the preprocess step you are suppose to only process that step once but the next step is for adding the number of batches and not for the text and labels.

go through link comment I shared. see if that helps, otherwise send me screenshot of the codes by personal DM.

8

I went through your comments in this and the other thread, and tried a few things, still about the same results. Sent screenshots by personal DM.

9

you seem to have completely mixed up the codes.

in the train val dataset the first step was to create train_size using the TRAINING SPLIT, and then you were suppose to split the text and labels, and further dividing them into training text, training labels, validation text and validation labels.

then creating the training dataset and validation dataset from the respective text and labels.

in the preprocess too you have added your own codes. this cell only had two code lines mentioning dataset for both step where one focused on the creating the dataset and the next dataset created the batches.

please get a fresh copy, redo your assignment. make sure only write codes as per instructions already given.

10

@houyao

in create model, just below the header grade function, you seem to have import tf.keras.regularizer.

please do not add, delete or edit anything outside the markers ### START AND END CODE HERE###, or you will end up submission failure even if your codes are correct.

11

Thanks. I’ll make some changes following your instructions.

12

get a fresh copy and then do, too many changes done.

13

Thanks. I got it working with a fresh copy. Thank you for your help, but I’m not exactly sure what the key point that made my original code not working.

14

according to your recent codes shared you are still hard coding the dataset.

here is a screenshot of the grade function without codes

Screenshot_20250216_101147_Chrome
Screenshot_20250216_101147_Chrome1080×837 161 KB

you do not require to again assign text and labels into the dataset as it was already done.

Also you do not require to separately assign the text and labels to its function recall and the. again use it in dataset.

you are suppose to define the dataset using the text and labels with their respective function recall in one single step.

the next dataset step mentions about the batch size.

15

Thanks for mentioning this. I have fixed the issue with your instructions by getting a new copy of the workbook.

  • Houyao Pan
16

Here is my last version of preprocess_dataset():

GRADED FUNCTION: preprocess_dataset

def preprocess_dataset(dataset, text_vectorizer, label_encoder):
“”"Apply the preprocessing to a dataset

Args:
dataset (tf.data.Dataset): dataset to preprocess
text_vectorizer (tf.keras.layers.TextVectorization ): text vectorizer
label_encoder (tf.keras.layers.StringLookup): label encoder

Returns:
tf.data.Dataset: transformed dataset
“”"

removed by mentor

return dataset

This one seems to be working, but the first part that creates the dataset is not in one line. Really appreciate your help.

  • Houyao
17

please don’t post codes here :ok_woman:t2:

it is still incorrect. read my previous comment carefully.

see the image I shared without codes which has only two code lines. you are suppose to only replace those None

18

Thank you. I thought I was replying in my email. I get that, thanks.

1 Like