Preprocess_input function in mobilenets and making inferences

Hi there!
I’m puzzled about CNN applications a lot, especially about the MobileNet programming assignment. Here are some of my questions:

  1. If we were to use this network for our own application, are we supposed to construct our own input data pipeline to feed in different images maybe like x-ray images or spectrograms? But most importantly, are we supposed to convert them to tensors or numpy arrays can do just fine?
  2. In the programming assignment, there is preprocessing stage. Could someone please explain me what exactly this function does? I tried checking the source code inside imagenet_utils.py but couldn’t find the answer.
  3. If I am using my own custom dataset that is comprised of numpy arrays, how can i make it work?
  4. The preprocessing function is used during training. So the inputs are already conditioned while making inferences during fitting. HOWEVER, when I want to evaluate my model after training, or when I want to make inferences, the data will not be preprocessed as in training. How should i go about doing it to make accurate inferences?
  5. I am trying to implement mobilenet for my own application and the accuracy cannot go beyond 18% for only 6 classes. But resnet is giving an accuracy that is close to 90%. (I am using min-max normalization on the entire train, test and validation set). Could you please tell me if this is a valid approach?

Cheers!

Briefly my thoughts here:

  1. You need to customize the pipeline for your data, tensors are used because of tensorflow and its libraries.

  2. Probably, you could use numpy or convert to tensors.

  3. You need preprocessing during inference too.

Thank you very much for your help but i am still puzzled.
1 & 2. It goes without saying that customizing the pipeline of the data is essential. However is it mandatory to use only tensors for keras applications like mobilenet, inceptionresnet, etc…?
3. I took a look at the mobilenet programming exercise once again, but I haven’t seen any part where validation data has been preprocessed as the training data. Am I missing something? I would expect the validation and inferences fail if training and validation data do not go through the same processing.
Cheers,

Hello @Ayse_Burcu_Ozkaptan,

  1. Check this out. Google for examples more relevant to your case :wink:

  2. All data should go through the same preprocessing steps that the training data has. In the assignment, I don’t find any step that will modify the training data hasn’t been applied to the validation data :wink: If you can’t find any such step either, that explains your observation. Be careful that the preprocess_input function was a part of the model and thus not exclusive to the training set, but any data that passes through the model.

Cheers,
Raymond

1 Like

Dear Raymond,
Thanks a lot for your valuable insights!

  1. I already tried it but then found it numpy was easier until yesterday i started having second thoughts about it. My second thoughts were focused on whether the functions of the applications can work on the numpy arrays or not, i.e. is it obligatory to use tensorflow datasets?
  2. Mind blown. Are you suggesting that the preprocess_input also works on the validation data?? If that is the case, that’s perfect and it clears half of my doubts, which brings us to my next question: Let’s assume that I trained my model and saved the model (weights). After loading the data and model, in order to make separate inferences on test data (or any data in the production environment), how should I proceed? should i run something like model.predict(preprocess_input(x_test)) or just model.predict(x_test)? The latter does not make much of a sense if the model has been trained on data that has a different input range. But I haven’t seen any sound examples on the correct use of model.predict() function either.

Cheers,

If you want to use TensorFlow and its built-in library of layers and models and optimizers, then you have to use tensors.

All of the portions of the data set must be statistically similar. First you pre-process all of the data, then you split it into training, validation, and test subsets.

Hello @Ayse_Burcu_Ozkaptan,

  1. I bet you have already had the answer - tensorflow dataset isn’t a requirement for using a tensorflow model. Again, google for how you may use different source of data. Hopefully you will find example code that can apply to your case, or through your googling, you will see a lot of different approaches. It is very usual for us to do that search ourselves and see what we can learn from the others.
  1. As I said, it was part of the model in that assignment, you don’t need to explicitly call it for your test data. Do you think it will help you if you go look at the notebook again and verify that it was a part of the model ;)?

Raymond

Dear Raymond,
Thanks a lot for your explanations! Now everything is a lot clearer.
Now in my own models I am paying special attention to train, dev and test sets and make sure i treat them the same way :slight_smile:
On one important note though for the fellow learners, it is of utmost importance to check what the model is expecting to receive. Some models expect the inputs to range from 0 to 255, whereas some others from -1 to 1. When i passed from mobilenet v2 to mobilenet v3, i thought they would have the same input parameters. But no… mobilenet v3 model has the input parameter “input_preprocessing” which is set to True by default which was renormalizing my already-normalized-data :smile: no wonder why it didn’t work :smiley:
Bottomline is, while using pre-trained networks, always always check if there is a mismatch between what the model is expecting and which sort of data you are feeding in and make sure that your test set is statistically similar to your train and dev sets.
Cheers,

You are welcome @Ayse_Burcu_Ozkaptan!

What you shared is very true! Because models are, afterall, engineered by human, just like any appliance, we would better read the instructions before use.

Cheers,
Raymond

1 Like