Cognitive/Cognitive Neuroscience Colloquium: Linking scalp ERPs to computational models of language and vision with multivariate pattern analysis

Colloquium | November 18 | 3-5 p.m. | 1104 Berkeley Way West

 Steven J Luck, University of California, Davis

 Department of Psychology

Linking scalp ERPs to computational models of language and vision with multivariate pattern analysis

Multivariate pattern analysis (MVPA) methods have become widespread in fMRI research, because they allow researchers to use the pattern of activation within a brain region to draw conclusions about the information being represented in that region. This approach is limited by the poor temporal resolution of fMRI, which makes it difficult to isolate initial feedforward sensory processing, feedback from higher areas, working memory maintenance, and decision processes. ERPs have the millisecond-level temporal resolution to isolate these different processes, but most researchers assume that the spatial resolution of scalp recordings is too poor to apply techniques such as decoding and representational similarity analysis (RSA). In this presentation, I will describe two studies demonstrating that these techniques can in fact be applied to address relatively subtle questions about the time course of neural representations. One study involves taking ERPs obtained while participants listened to stories and linking them to computational models of natural language processing. The other involves taking ERPs obtained while participants viewed photographs of natural scenes and linking them to a computational model of saliency, to spatial maps of semantic richness, and to a deep convolutional neural network model of scene classification.

 CA, annecollins@berkeley.edu, 5106647146