How to pass 2 datasets to .fit?

Course Q&A TensorFlow Developer Professional Natural Language Processing in TensorFlow
1

Hi! I’ve loaded two datasets with sentences and labels from txt file with TextLineDataset, vectorized them by .map. Now I need to pass this data to model.fit. And I’ve found, that I can’t pass datasets directly to model as x and y. I need to do .as_numpy_iterator() or convert to dataset in format like ((all_first_dataset_elements), (all_second_dataset_elements)), that is not easy task. So, by my opinion both options are a bit ugly. I am sure, that should be more easy and logical way to use built-in datasets. Can you give me idea how to do it as it have to be?

2

See Dataset.zip

3

I know about it? I can even say, that I have zipped elementwise x,y data. Should it be helpful?

4

What do you mean by?

You can pass a dataset directly to a model. See x in Args section of fit

5

I don’t know, I have error like

model = keras.Sequential([
    tf.keras.layers.Input(shape=(None, length_with_padding)),
    tf.keras.layers.Dense(9),
    tf.keras.layers.Softmax()
])

model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy'])

model.fit(tf.data.Dataset.zip(test_sentences_vec, test_tags_vec), epochs=3)

ValueError: Exception encountered when calling layer 'sequential_3' (type Sequential).
    
    Input 0 of layer "dense_3" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (None,)
    
    Call arguments received by layer 'sequential_3' (type Sequential):
      • inputs=tf.Tensor(shape=(None,), dtype=int64)
      • training=True
      • mask=None

>>> list(test_sentences_vec.take(10).as_numpy_iterator())
[array([1]),
 array([1]),
 array([1]),
 array([1]),
 array([ 2186, 22658,     1,  3508,  1032,  3173,  1516,     1,  1506,
            1]),
 array([    1, 25050,  1516, 27993]),
 array([   1, 1075]),
 array([1]),
 array([1]),
 array([1])]
6

This trace shows the problem:

2 things to notice:

  1. The model requires you to provide a 3d tensor as each input since input_shape has 2 dimensions.
  2. Padding should be done before training the model.

Here’s an example of a 3D post padded input with batch size of 2:

array([[[    1],
        [    0],
        [    0],
        [    0],
        [    0],
        [    0],
        [    0],
        [    0],
        [    0],
        [    0]],

       [[ 2186],
        [22658],
        [    1],
        [ 3508],
        [ 1032],
        [ 3173],
        [ 1516],
        [    1],
        [ 1506],
        [    1]]])

Since you’re dealing with text data, you should check the following:

  1. Is it correct to pass a 3D input to the model.
  2. Since the numbers in the sequence stand for token IDs and don’t mean anything, are you missing a step in processing the inputs before any Dense layers.
1 Like
7

hmm, ok, thanks. Will try to check padding.

3D input? What do you mean? Where is it?

And can you help me is anything equivalent of padding generation individually per batch in tensorflow, like that I asked here?

8

As far as 3D input is concerned, the line below mentions that the model will be capable of processing 3D inputs:

For a 2D input, the correct line is like this:
tf.keras.layers.Input(shape=(length_with_padding, )),

Remember 2 things as far as specifying input shape to the model is concerned:

  1. Pass the shape of each sample excluding the batch dimension.
  2. None means that you don’t know how many steps will be processed during a single forward pass. Please complete courses 3 and 4 to better understand the need for a 3D input.

I’m not a mentor for NLP and so have added a few mentors on the other topic.

9

Is this not 2D input? tf.keras.layers.Input(shape=(None, length_with_padding)), Why is syntax so strange in this case? It really more looks like 2D

10

Here’s what shape=(None, length_with_padding) means:

None here means that you don’t know how many timesteps the model is going to process but each timestep has length_with_padding features.

Since you have a background on deep learning, I’m assuming you understand how RNN layers work, what the RNN outputs are and why None makes the model more flexible.

11

I just say about syntax. Usually 3 dimensions are defined as 3 numbers in shapes brackets, as I remember

12

Tensorflow always skips the batch dimension and requires you to specify shape of 1 input sample. If this is news to you, please go back to other notebooks and view the input_shape parameter.

1 Like