Clustering and outliers dropout as regularization method

Could be possible to use a cluster analysis of the training/dev to regularize the learning algorithms based on the assumption that the data that is statistically out of the regular data may prevent overfitting? This hypothesis may not be preferred for image recognition, but my field of study is SCADA data, I would think that I set of data that is part of a rare cluster or if outliers may not be good as training data.

muy buena informacion