Hi @Menelaos_Gkikas,
you can conduct z-normalization regardless of whether the variable is normally distributed or not.
E.g. image if you have several features w/ a Student t-Distribution which have fatter tails than a normal distribution.
Still after scaling w/ z-normalization you have made them comparable which is beneficial for your algorithm: After all, you want to make sure that your features are in a comparable, reasonable way to have a nice training process and run gradient descent more effectively without biasing the algorithm to high magnitude features, see also: Questions on normalizing really huge data - #2 by Christian_Simonis
It you have really strange distribution shapes, one would have to think about if other scaling approaches might be better (like min/max or so)…
Hope that helps! Please let me know if you have any further questions.
Best regards
Christian