Getting nan losses while training the model

I was trying to train a model for kaggle’s spaceship titanic dataset. Spaceship Titanic | Kaggle. But while training the model the losses I am getting are nan values. Can anyone please let me know what mistake I am making.

The following code is after I have preprocessed the dataframe and converted it to tensor.

norm_l = tf.keras.layers.Normalization(axis=-1)
norm_l.adapt(x_train) # learns mean, variance
Xn = norm_l(x_train)

model=Sequential([
tf.keras.Input(shape=(15,)),
Dense(units=128,activation=‘relu’),
Dense(units=64,activation=‘relu’),
Dense(units=32,activation=‘relu’),
Dense(units=16,activation=‘relu’),
Dense(units=4,activation=‘relu’),
Dense(units=2,activation=‘relu’),
Dense(units=1,activation=‘sigmoid’)
])

model.compile(
loss=tf.keras.losses.BinaryCrossentropy(),
optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
)

history = model.fit(
Xn,y_train,
epochs=40
)

And this is the output I am getting.
Epoch 1/40
182/182 [==============================] - 1s 1ms/step - loss: nan: 0s - loss: n
Epoch 2/40
182/182 [==============================] - 0s 1ms/step - loss: nan
Epoch 3/40
182/182 [==============================] - 0s 1ms/step - loss: nan
Epoch 4/40
182/182 [==============================] - 0s 1ms/step - loss: nan
Epoch 5/40
182/182 [==============================] - 0s 1ms/step - loss: nan
Epoch 6/40
182/182 [==============================] - 0s 1ms/step - loss: nan
Epoch 7/40
182/182 [==============================] - 0s 1ms/step - loss: nan

It probably has to do with this part here:

norm_l = tf.keras.layers.Normalization(axis=-1)
norm_l.adapt(x_train) # learns mean, variance
Xn = norm_l(x_train)

Try removing and see what you get.

Still getting nan losses

Hi @Ayush_Aman,
Please try to run the following to see if there is any np.nan value or infinity value in your dataset.

print('x_train', np.sum(np.isnan(x_train)))
print('Xn', np.sum(np.isnan(Xn)))
print('y_train', np.sum(np.isnan(y_train)))

print('x_train', np.sum(np.isinf(x_train)))
print('Xn', np.sum(np.isinf(Xn)))
print('y_train', np.sum(np.isinf(y_train)))

Raymond

x_train 588
Xn 29120
y_train 0
x_train 0
Xn 0
y_train 0

Yes there are null values. Now I’m even more confused. If I check for the null values just after reading the dataframe this is the output I get.

In- train.shape
Out- (8693, 17)

In-train.isnull().count()
Out-
PassengerId 8693
Age 8693
RoomService 8693
FoodCourt 8693
Spa 8693
VRDeck 8693
Transported 8693
HomePlanet_Earth 8693
HomePlanet_Europa 8693
HomePlanet_Mars 8693
CryoSleep_False 8693
CryoSleep_True 8693
Destination_55 Cancri e 8693
Destination_PSO J318.5-22 8693
Destination_TRAPPIST-1e 8693
VIP_False 8693
VIP_True 8693
dtype: int64

Please try train.isnull().sum() or train.count()instead.

1 Like

Yes, this was the problem. There were some null values removing which my neural network is working fine. Thank you @rmwkwok and @gent.spah

1 Like

You are welcome @Ayush_Aman!

Cheers,
Raymond