Abstract
Convolutional Neural Networks are the state-of-the-art for object recognition algorithms. However, little is known about the internal representations they build during training, and how internal features relate to object classification. In this work, we examined the color tuning of units in hidden layers of the well known AlexNet and their relevance for the successful recognition of an object. We first selected the patches for which the units are maximally responsive among the 1.2M images of the training dataset. Then, we segmented the object being part of the patch from its surroundings, using a k-mean clustering algorithm on the color distribution within the patch. Then we independently varied the color of the object and surroundings to extract the unit's chromatic tuning. We found that in the convolutional layers, 1 through 5, over 36% of the units are sensitive to either the color of the object or its surroundings, reaching over 50% in the fifth layer. Half of these units were sensitive to both object and surround color. In addition, we observed that modifying the color of either object or surround greatly affected the accuracy of the network for recognizing objects. For 74% and 79% of the color tuned units in layer 5, changes of the object color and the surroundings color respectively led to wrong classifications. Our approach allows us to investigate color tuning and object recognition simultaneously and for images the network nodes strongly respond to. Our results show that color plays an important role within AlexNet. Both the color of the object and the surroundings contribute similarly to object recognition.
Meeting abstract presented at VSS 2018