I need advice in LSTM Model

I run kaggle store forecast for training
I have the following
model_lstm = Sequential()
#it use to be 190
model_lstm.add(LSTM(50, activation=‘tanh’, return_sequences=True,input_shape=(X_train_series.shape[1], X_train_series.shape[2]), kernel_regularizer=regularizers.l2(0.001))) # Use n_steps_in and n_features here
model_lstm.add(LSTM(250, activation=‘tanh’))
model_lstm.add(Dense(250, activation=‘relu’)) # Additional Dense layer
model_lstm.add(Dense(50, activation=‘relu’)) # Additional Dense layer
#it used to be 0.0001
model_lstm.compile(loss=‘mse’, optimizer=optimizers.Adam(learning_rate=0.001, clipnorm=1.0))


and i found out by running fit inside a for loop performs better
for i in range(1,140):
lstm_history = model_lstm.fit(X_train_series, Y_train,
validation_data=(X_valid_series, Y_valid),

I need guidance on what should i do Store Item demand forecasting | Kaggle to reduce the underfitting in particular cases

If you’re going to post your code and ask for help, please use the “preformatted text” tag. This will allow the indentation to be displayed correctly.