Windowed_dataset function Slicing Problem

[Filip1988]

Hello,

i am trying to understand code logic of function to get windowed dataset. I started to do excercises in colab “C4_W4_Lab_1_LSTM.ipynb” where i found in the code something that i really don’t understand.

see bellow code to get window dataset from colab:

def windowed_dataset(series, window_size, batch_size, shuffle_buffer):

series = tf.expand_dims(series, axis=-1)
ds = tf.data.Dataset.from_tensor_slices(series)
ds = ds.window(window_size + 1, shift=1, drop_remainder=True)
ds = ds.flat_map(lambda w: w.batch(window_size + 1))
ds = ds.shuffle(shuffle_buffer)
ds = ds.map(lambda w: (w[:-1], w[1:]))
return ds.batch(batch_size).prefetch(1)

what i don’t understand, why in a step : ds = ds.map(lambda w: (w[:-1], w[1:])): we are slicing dataset as followed w[:-1], w[1:]), to be more accurate, why we are getting y value (label) as a slice w[1:] , should not be this w[-1:] instead?

when we slice this as w[1:] we are getting labels with the same shape as we have feature values…
lets say we have dataset [1,2,3,4,5,6]
after slicing we get:
feature window [1,2,3,4,5]
label:[2,3,4,5,6]

should not we rather get sliced window and label by w[-1:] as followed?
feature window [1, 2, 3, 4, 5]
label:[6]

In previous weeks of this course, there were used in code logic target labels either w[-1:] or w[-1], therefore i am confused now, since i thought that each label should be just one target value that should be predicted based on respective featured window to that label.

[german.mesa]

Hi @Filip1988,

I was struggling at first to understand the logic.

So, I decided to create a separated code to play a little bit. It’s a completely artificial dataset where I printed results by phase:

SAMPLE_SIZE = 10
BATCH_SIZE = 4
WINDOW_SIZE = 5

# Creating a simple tensorflow dataset
dataset = tf.data.Dataset.from_tensor_slices(tf.range(SAMPLE_SIZE))

# The window() method returns a dataset containing windows, where each window is
# itself represented as a dataset
print("Getting windows datasets...")
dataset = dataset.window(WINDOW_SIZE, shift=1, drop_remainder=True)
for window in dataset:
    print(f"Dataset...{window}")
    print([elem.numpy() for elem in window])

# The flat_map() method returns all the tensors in a window dataset
print("\nFlat map over each window dataset...")
dataset = dataset.flat_map(lambda window: window.batch(WINDOW_SIZE))
for window in dataset:
    print(f"Dataset...{window}")

# Must be equal or bigger than dataset length for perfect shuffling
print("\nShuffling window dataset...")
dataset = dataset.shuffle(SAMPLE_SIZE)        

# I would like to use the last value as the target. As each window is a tensor:
#  window[:-1] : all the elements but last
#  window[-1:] : last element
print("\nSplitting features & target...")
dataset = dataset.map(lambda window: (window[:-1], window[-1:]))

for x, y in dataset:
    print("Input:", x.numpy(), "Target:", y.numpy())

# Combines consecutive elements of this dataset into batches
dataset = dataset.batch(BATCH_SIZE, drop_remainder=True)

# Allows later elements to be prepared while the current element is being processed
dataset = dataset.prefetch(1)                     

print("\nPrint batches...")
for x, y in dataset.take(2):
  print("Input:\n", x.numpy())
  print("Target:\n", y.numpy())

Hope it helps,

[Filip1988]

thank you very much for explanatory code, it is very well explained what is happening step by step just when i looked at video in Course 4 Week 4 - Bidirectional LSTMs, i guess there is missing “-” in the code where labels are sliced by w[1:] ? is it correct? or when feed to lstm it could be sliced like this since target value is the last one in that sliced label vector?

[german.mesa]

Hi @Filip1988,

When you use expand_dims with negative value for axis, you are adding a new dimension at the end of the tensor. Imagine you have a 24x24x3 tensor, then it will become a 24x24x3x1. In our case, we have a one dimensional array with shape [3650], now it turns to be [3650,1] tensor. So far so good.

About second question, I’d recommend to change w[1:] by w[-1:], as it will be easy to understand. I believe it’s a typo - although your model would probably train anyway.

Best,

[Filip1988]

I understand, thank you