Geoffrey M. Boynton
Professor
Department of Psychology
University of Washington
PO Box 351525

Office:  Guthrie Hall, Room 233A
Phone: (206) 685-6493
Fax:      (206) 685-3157
email:   gboynton@u.washington.edu

Research Statement

The primary focus of my research has been on the effects of attention on the representation of stimuli in the human visual cortex. 20 years ago, when I was a graduate student starting my career in cognitive neuroscience, it was widely held that the early stages of visual processing were part of a feed-forward processing stream that was unaffected by ‘top-down’ factors such as attention. Indeed, beginning with the pioneer work by Hubel and Wiesel, most electrophysiological studies of these early stages, such as the primary visual cortex (V1), were conducted on anesthetized cats and monkeys with the assumption that results would generalize to the awake, behaving state.

However, over the past two decades we’ve learned that what and where an observer is attending has a profound influence on the neuronal response to incoming visual stimuli. This is a remarkable shift in our understanding of the functional organization of the mammalian visual system. It means that attention alters the brain’s interpretation of a stimulus as soon as it enters the cortex. While it was once thought that areas such as V1 maintained a veridical, movie screen-like representation of the visual scene, we now know that all cortical areas in the visual system are affected by what and where you are attending. Our brain’s representation of the distal world is unavoidably altered by what we are trying to see.

Much of our knowledge about how attention alters neuronal responses in visual cortex comes from functional MRI (fMRI) studies in human subjects.  As I will describe below, much of my work on attention has involved this method. 

Research on the response properties of the fMRI signal
Functional MRI (fMRI) was in its infancy when I started my postdoc at Stanford University with David Heeger in 1994. It had been only a few years since the original papers were published showing that a blood oxygen level dependent (BOLD) signal could serve as a correlate of brain responses using standard clinical MRI scanners. The method was so new that almost nothing was known about the relationship between the BOLD signal and the underlying neuronal response. David Heeger and I wanted to apply fMRI to study the human visual system, but before we could get started we decided to run a series of experiments to determine if we were working with a well-behaved measure of neuronal activity. Specifically, we wanted to see if the BOLD response behaved linearly in time so that, for example, the response to two successively presented stimuli could be predicted from the response to single stimuli alone. To our surprise, we found that the BOLD signal was remarkably linear. This greatly simplified the interpretation of fMRI results – the linear model is the backbone of nearly all fMRI analysis software packages. Our publication of this result in the Journal of Neuroscience in 1996 formed the justification for nearly all fMRI analysis methods today (Boynton, Engel et al. 1996). More importantly, it meant that we could proceed with our original plans to investigate the human visual system with fMRI.

I have since maintained an interest in the ‘hemodynamic coupling problem’ and have published work on the effects of adaptation (Finney and Boynton 2003) and transients (Tuan, Birn et al. 2008) on the fMRI signal. I have also enjoyed writing a series of commentaries and reviews on the topic (Boynton 2003; Boynton 2005; Krekelberg, Boynton et al. 2006; Boynton 2011).

Effects of spatial attention in primary visual cortex (V1)
Around this time, the first studies were published showing that attention could affect the neuronal responses in higher areas of the macaque visual cortex such as area V4 (Desimone and Duncan 1995) and MT (Treue and Maunsell 1996). It was natural to apply fMRI to see if we could find effects of spatial attention in the human visual cortex. We were surprised to find robust effects of spatial attention not only in higher visual areas, but also in V1 (Gandhi, Heeger et al. 1999). This result was so novel that we had some difficulty getting our results through the review process, but in the end two other laboratories had just discovered the same result (Martinez, Anllo-Vento et al. 1999; Somers, Dale et al. 1999). Since then, effects of attention on fMRI responses in V1 have been published hundreds of times (many in my own laboratory); this is now literally textbook knowledge.

A curious fact is that the effects of spatial attention on the BOLD signal in human V1 appear to be larger than what is expected from monkey electrophysiology studies, especially for weak or low contrast stimuli. My research back at the Salk Institute showed that the strength of these attentional effects did not depend on the strength, or contrast, of the physical stimulus (Buracas and Boynton 2007). This apparent discrepancy between human fMRI and monkey electrophysiology results has been an ongoing topic of my research. One intriguing explanation is that because the BOLD signal presumably reflects an aggregate response over a large pool of neurons, the BOLD signal may actually be more sensitive than single-unit measures. I’ve discussed this possibility and a variety of other possible explanations in a recent review (Boynton 2011).

Feature-based attention
Attention can be directed to locations in space (spatial attention, described above), or to different features such as toward directions of motion, or colors (feature-based attention). Early single-unit studies in macaque showed that attention to a specific feature enhanced the responses in neurons that are selective to that feature, and suppressed the response in neurons selective away to the attended feature. These effects of feature-based attention can be found in the responses of neurons with receptive fields far away from the spatial focus of attention (Treue and Martinez Trujillo 1999).

We were able to find the first effects of such global feature-based attentional effects in humans using fMRI (Saenz, Buracas et al. 2002). We measured the fMRI response to an unattended moving stimulus while subjects attended to either a matching or un-matching direction in the opposite hemifield. Consistent with the electrophysiological study, we found a greater fMRI response to the unattended stimulus when it matched the direction of motion attended elsewhere. This ‘global feature-based’ attentional effect was found all over the visual cortex, including area V1. We also found that it applies to color so that attention to a color (say, green) on one side of the visual field enhances the responses to all stimuli in the visual field sharing the attended color, regardless of the spatial focus of attention. This mechanism has implications for tasks such as visual search which is greatly benefited by knowing the feature of an object that you are looking for.

In a later study, we applied a new method of fMRI data analysis called ‘multi-voxel pattern analysis’ (MVPA) to show that feature-based attention affected the pattern of responses across voxels in early visual areas, including V1, even in the absence of a stimulus (Serences and Boynton 2007). That is, just like for spatial attention, feature-based attention appears to modify neuronal responses in a way that is independent of the physical stimulus. One interpretation of these results is that spatial and feature-based attention is modulating baseline neuronal responses in the anticipation of an incoming visual stimulus, perhaps setting up the network to be particularly sensitive to incoming visual stimulation that matches the attended locations and features.

Automatic processing of unattended information
The flip side of studying the neuronal representation of attended stimuli is to see what happens to the rest of the unattended visual field.  Recent work in my lab is showing that an unattended stimulus can slip through the attentional filter if it is threatening or if it shares features or temporal synchrony with an attended stimulus.  For example, we have found that when a brief flash of color directs attention to one location in the visual field, subjects are better at detecting a subsequent target anywhere in the visual field as long as it has the same color as the cue (Lin, Hubert-Wallander et al. 2011).  Also, when viewing a looming object on a computer screen, the object automatically attracts attention to its location only if it is on a collision course with the subject’s head.  Amazingly, this automatic capture of attention is sensitive to imperceptible changes in the trajectory of the looming object (Lin, Murray et al. 2009). Finally, we have discovered that unattended information in the peripheral field can be passed into memory if it occurs in time with a foveally presented target.  We call this a ‘screen-capture’ mechanism that grabs all information in the visual field at behaviorally relevant points in time (Lin, Pype et al. 2010).  All of these discoveries have been made through behavioral measurements in the lab.  We are now pursuing the neuronal basis of these attentional effects using fMRI and EEG techniques.

Modeling effects of attention
Our understanding of the effects of attention on cortical responses is mature enough that a few general principles have emerged. This has allowed for the generation of computational models that capture these principles. In my research, I have summarized the results of behavioral, single-unit and neuroimaging studies on spatial and feature-based attention in the context of simple computational models (Boynton 2004; Boynton 2005; Boynton 2009; Boynton 2011). These models build upon existing models of stimulus-driven visual processing and add on simple mechanisms for how top-down attention modifies these incoming signals. They have proved to be useful to not only summarize the existing body of research on attention but they also make testable predictions that have allowed the field to move ahead in a systematic manner.

Current and future research on divided attention
The research described above is on selective attention in which attention is directed to one location or feature at a time while ignoring all other parts of the visual field. In everyday life, however, we regularly have to divide our attention between stimuli at different spatial locations or between multiple features or objects. While there is a deep literature on how divided attention affects behavioral responses, but much less is known about how divided attention modulates neuronal responses in sensory cortex.

My most current research has moved toward understanding the neuronal mechanisms of divided attention in the visual system. For example, very recent work of mine, in collaboration with Dr. John Palmer in our department has shown that it is possible for subjects to divide attention between two spatial locations with no behavioral cost compared to attending to a single location alone. We have also found a complementary fMRI result showing that the effects of attention in V1 for divided spatial attention are just as strong as for selective attention (Runeson, Palmer et al. in preparation). On the other hand, divided attention across features comes at a cost. We have shown that it is more difficult to attend to two opposing directions of motion or colors at the same time than to attend to two matching directions or colors (Saenz, Buracas et al. 2003). Similarly, it is much easier to divide attention between the color and the motion of a single surface than to attend to the color of one surface and the motion of another (Ernst, Palmer et al. 2012). A complementary neuroimaging study that is in review shows that this may be the result of an automatic spread of feature-based attention to all properties of an object (Ernst, Boynton et al. in review).

I have recently discovered that existing computational models for selective attention (Boynton 2009; Reynolds and Heeger 2009) can be easily modified to predict these behavioral and neuroimaging results on divided attention. Specifically, it turns out that attending to two features comes at a cost if the neurons selective to the two features share a common suppressive normalization pool. I find this exciting because to date, there are no mechanistic descriptions for why we sometimes can and sometimes cannot split our attention to two locations, features, or objects. Linking divided attention effects to a normalization process may generalize to a range of domains including auditory attention, tactile attention and even general cognition because normalization is believed to be a general principal of neuronal processing in the brain (Carandini and Heeger 2012). I believe that there are strong practical applications for this discovery. In today’s over-stimulating world, the ability to ‘multitask’ has been considered a great virtue. However, a growing body of research is showing that we are not nearly as good as multitasking as we subjectively think (Marois and Ivanoff 2005).

References
Boynton, G. M. (2003). Understanding the Neuronal Basis of the fMRI Signal. Modulation of Neuronal Signaling: Implications for Visual Perception. G. T. Buracas, T. D. Rukensas, T. D. Albright and G. M. Boynton. Amsterdam, IOS Press 334.
Boynton, G. M. (2004). "Adaptation and attentional selection." Nat Neurosci 7(1): 8-10.
Boynton, G. M. (2005). "Attention and visual perception." Curr Opin Neurobiol 15(4): 465-469.
Boynton, G. M. (2005). "Imaging orientation selectivity: decoding conscious perception in V1." Nat Neurosci 8(5): 541-542.
Boynton, G. M. (2009). "A framework for describing the effects of attention on visual responses." Vision Res 49(10): 1129-1143.
Boynton, G. M. (2011). "Spikes, BOLD, attention, and awareness: a comparison of electrophysiological and fMRI signals in V1." J Vis 11(5): 12.
Boynton, G. M., S. A. Engel, et al. (1996). "Linear systems analysis of functional magnetic resonance imaging in human V1." J Neurosci 16(13): 4207-4221.
Buracas, G. T. and G. M. Boynton (2007). "The effect of spatial attention on contrast response functions in human visual cortex." J Neurosci 27(1): 93-97.
Carandini, M. and D. J. Heeger (2012). "Normalization as a canonical neural computation." Nat Rev Neurosci 13(1): 51-62.
Desimone, R. and J. Duncan (1995). "Neural mechanisms of selective visual attention." Annu Rev Neurosci 18: 193-222.
Ernst, Z., R., J. Palmer, et al. (2012). "Dividing attention between two transparent motion surfaces results in a failure of selective attention." J. Vis.
Ernst, Z. R., G. Boynton, et al. (in review). "The spread of attention across features of a surface." Journal of Neurophysiology.
Finney, E. M. and G. M. Boynton (2003). "Orientation-specific adaptation in human visual cortex." J. Neurosci In Press.
Gandhi, S. P., D. J. Heeger, et al. (1999). "Spatial attention affects brain activity in human primary visual cortex." Proc Natl Acad Sci U S A 96(6): 3314-3319.
Krekelberg, B., G. M. Boynton, et al. (2006). "Adaptation: from single cells to BOLD signals." Trends Neurosci 29(5): 250-256.
Lin, J. Y., B. Hubert-Wallander, et al. (2011). "Rapid and reflexive feature-based attention." J Vis 11(12).
Lin, J. Y., S. O. Murray, et al. (2009). "Capture of attention to threatening stimuli without perceptual awareness." Curr Biol 19(13): 1118-1122.
Lin, J. Y., A. D. Pype, et al. (2010). "Enhanced memory for scenes presented at behaviorally relevant points in time." PLoS Biol 8(3): e1000337.
Marois, R. and J. Ivanoff (2005). "Capacity limits of information processing in the brain." Trends Cogn Sci 9(6): 296-305.
Martinez, A., L. Anllo-Vento, et al. (1999). "Involvement of striate and extrastriate visual cortical areas in spatial attention." Nat Neurosci 2(4): 364-369.
Reynolds, J. H. and D. J. Heeger (2009). "The normalization model of attention." Neuron 61(2): 168-185.
Runeson, E., J. Palmer, et al. (in preparation). "Unlimited capacity for divided attention in human primary visual cortex."
Saenz, M., G. T. Buracas, et al. (2002). "Global effects of feature-based attention in human visual cortex." Nat Neurosci 5(7): 631-632.
Saenz, M., G. T. Buracas, et al. (2003). "Global feature-based attention for motion and color." Vision Res 43(6): 629-637.
Serences, J. T. and G. M. Boynton (2007). "Feature-based attentional modulations in the absence of direct visual stimulation." Neuron 55(2): 301-312.
Somers, D. C., A. M. Dale, et al. (1999). "Functional MRI reveals spatially specific attentional modulation in human primary visual cortex." Proc Natl Acad Sci U S A 96(4): 1663-1668.
Treue, S. and J. C. Martinez Trujillo (1999). "Feature-based attention influences motion processing gain in macaque visual cortex." Nature 399(6736): 575-579.
Treue, S. and J. H. Maunsell (1996). "Attentional modulation of visual motion processing in cortical areas MT and MST." Nature 382(6591): 539-541.
Tuan, A. S., R. M. Birn, et al. (2008). "Differential transient MEG and fMRI responses to visual stimulation onset rate." International Journal of Imaging Systems and Technology 18(1): 17-28.