9

I trained a convolutional network to classify images of a mechanical component as good or defective. Though the test accuracy was high, I realized that the model performed poorly on images which had slightly different lighting.

The features that I'm trying to detect are subtle, and lighting seems to trip the model into calling a lot of false negatives. Most, if not all, of the training and test data came from images which were shot under diffused lighting. The new images that I tried the model on were shot with focused lights.

Hoping that histogram equalization(CLAHE) would help, I did equalization on images before feeding it to the model, did this for training as well as test data. The problem then was that accuracy was high, but the model seemed to have learnt some other correlation between the images and labels. After equalization, everything in the image stands out, the defects become even more subtle and hard to detect even for human eye.

The model prediction changed even when the same component was shot with the same lighting but with the light at different orientations. So my question is, what can I do, either at the data end or with the model, so that it becomes robust to changes in lighting(or same kind of lighting but changes to orientation of lighting)?

Effective_cellist
  • 191
  • 4
  • Is it feasible to collect more data with variations in lighting? The difference between diffuse and direct lighting in a scene will be difficult to simulate or allow for in pre-processing. – Neil Slater Sep 03 '17 at 15:45
  • @NeilSlater Yes, I can collect more data. I am more interested in making the the model robust to illumination changes, to a reasonable extent. One thing I noticed was that, the model prediction changed even when the same component was shot under the same lighting, but different orientations. Making the model robust to different kinds of lighting may be hard, are there techniques to make it robust to different orientations of same lighting? – Effective_cellist Sep 03 '17 at 16:03
  • Subtracting the mean from the images often helps. Also, how well is your data balanced? If in the training dataset 1% is class 1, the model will not learn anytjing and classify all as class 0. – Alex Sep 07 '17 at 08:42
  • @Alex I haven't tried mean subtraction. Thanks for the suggestion. Also, my dataset is well-balanced. I have 2 classes and each account for 50% of the data. – Effective_cellist Sep 07 '17 at 11:08

1 Answers1

2

It's called overfitting, your model learned to predict labels based on features which are not actually responsible for particular predictions, so when they are not presented it fails to predict right. Although there are various ways to deal with overfitting (e.g. Dropout) what you seem to need is image augmentation. It is a simple yet very powerful way to train a robust neural network. For your case - just simulate different lighting conditions by e.g. increasing/decreasing pixel values randomly for your inputs for same labels. It's also common practice to add random noise, random rotations, flip etc.

For more info check this stackoverflow answer.

Also, I recently published project where I used various of Augmentation functions which you may find useful. See: https://github.com/Naurislv/P12.1-Semantic-Segmentation/blob/master/augmentation.py

Nauris Dorbe
  • 21
  • 3