Bharathi Jagadeesh: How the brain sees pictures: neural selectivity for realistic pictures in the temporal lobe

Abstract

Neurons in perirhinal cortex (PRh) of the macaque monkey respond selectively to images of objects, faces, and scenes. Single PRh neurons can respond indistinguishably to multiple stimuli in a set of 16-24 images. It is widely believed that the images which elicit indistinguishable responses must be similar in some visual dimension. However, since the physical space these images occupy is multi-dimensional, quantifying the physical similarity among them is difficult. One approach to ranking the similarity of pairs of images is the Earth Mover Distance metric, (EMD) which was developed to search image databases. When queried with one natural image, the EMD uses the color content of the images and returns those that are most similar to the target. The images returned have a high-level similarity to the target, as qualitatively judged by human observers (Rubner, Y 1998). We used the EMD to assess physical similarity of stimuli that produce equivalent responses in single PRh neurons. We categorized stimuli into effective and ineffective groups; the effective and ineffective stimuli elicited responses that were indistinguishable from the maximum and minimum response (respectively) of the neuron. As assessed by the EMD, the pairwise similarity between images within the effective group was greater than the pairwise similarity of images across effective and ineffective groups (p<.001). Neural selectivity in PRh and the EMD may both reflect the underlying perceptual similarity of these realistic pictures.