Why is my multi-task-learning model not working?

I’m writing Python code to predict fetal head circumference using regression and classification together in a single model.

The model will train to classify a fetal head image into a range (e.g., 50–60 mm) representing its circumference.

If you look at the code, you’ll notice I calculate the loss for both guessing a circumference value and for classifying the image into a circumference range. (My initial plan was to guess the range based on the output of the circumference the model guesses, but I am still not sure how to implement that, so the current model guesses these separately based on the CNN filters.)

However, I’m not experienced with the MTL model, and for some reason, the loss I get for the classification is always 3.5 and not improving.

Here is a reference to my GitHub