Transfer Learning to deal with system error

One key note I captured in this course is that deep learning is not good at dealing with system noise. In my project, i want to accomplish this:

  1. Assume system A is well trained using a lot of data (X_train, Y_train), which is costly and time-consuming. The J (cost) of training and validation is low and meet the product specs. In the field, this product is used to take measurement (X) and output the results (Y) using the model trained.
  2. In manufacturing, i need to make thousands and millions of system that is the same as A, but of course, not exactly the same as A due to manufacturing noises though within process tolerance.
  3. I don’t have enough budget and time to retrain A1A2A3… in production. How do I design a process to let A1, A2, A3, … quickly learn and adjust the model slightly to meet the product specs?
  4. Note: we don’t expect the model will change largely for A1, A2, … If so, that means the product Ai in manufacturing line is propobably out of specs already, thus should be considered as yield loss.

My thinking of dealing with issue:

  1. Use the trained model of A as baseline
  2. Test a few standard training examples (limited number of examples) and use those examples to train A1, A2, … using transfer learning

Is my thought process correct? Any guidelines or suggestions?

Thank you!

Why do you feel this is true?

I should have said “system error”. In " Cleaning Up Incorrectly Labeled Data", Prof. said " They (deep learning algo.) are less robust to systematic errors." He further gives an example of always labeling a white dogs as cat.
In context of my project, System A1 (or A2, A3, …) in production, unlike system A, could have an input X1, which didn’t lead to Y1 due to system error. This system error may be due to the manufacturing error. In that case, I will not be able to use the model trained in system A and apply it to system A1 and following systems in production.

Sorry, but I do not understand what systems A1, A2, and A3 represent in your example.

Sorry if i didn’t make it clear.
Say, I make a sensor A, which can measure some physical parameters X. I use deep learning to train the sensor A, so that it can predict Y with the input X. To train it, it will need a lot of data of (X_train, Y_train). Assume it is trained well and it did a pretty good job in estimating Y for a measurement X. (note: this process will be very costly and time-consuming)
Now, I need to use the same manufacturing process to make many other sensors A1, A2, A3, … I could just use the same model of A for A1 take a measurement and predict.
The problem is that, A1, A2, A3, … are manufactured at different time although the manufacutring process is the same as A. It means, there will be system errors due to manufacture process. Using the same model trained on A could give incorrect estimate while using A1, A2, …
My original question is whether my steps will be valid process to resolve the system errors in A1, A2, … introduced in manufacturing process.

If your original data set X_train contains enough data, it will include the normal variations due to manufacturing tolerances and measurement error.

If it doesn’t have enough variation, you can artificially create additional examples by adding noise or variance to the examples you already have. It’s a very common method of “stretching” a limited data set.

This data set is used to train your model A. You can then use it for predictions using new data.

So I do not understand the need for A1, A2, and A3.

A1, A2, A3, … are just copies of A that are sold to the customers and customer expect A1, A2, A3… they purchased should be all the same or within the specs defined by datasheet. Manufacturing error I mentioned has nothing to do with the training set. It occurs when producing A1, A2, A3, …

In traditional manufacturing process, process tolerance can be defined and derived by product spec. For example, a speed sensor could just use v = distance / time and measure the distance and time will give velocity. We can clearly know how the errors in distance and time measurement can impact velocity it shows to the customer.

However, if a product will give a result (Y) that is infered by the measured data (X) using Deep learning model, there will be a problem. Because we don’t know how the manufacturing variations (in producing A1, A2, A3…) could impact the accuracy of Y using the model trained on A. We only know that the manufacturing errors could be small and we may be able to fine tune the model to adapt the manufacturing errors.

I know this is confusing as it’s very different from many cases we discussed in this course. I hope it’s now clear. Thank you a lot for your time and patience.

Include all the expected variations in your training set, and the issue is resolved. All your customers can use model A.