Toward Developmentally Reasonably Self-Supervised Learning: IHD/Developmental Colloquium Fall 2019

Colloquium | October 7 | 12:10-1:30 p.m. | 1104 Berkeley Way West

 Dan Yamins, tanford University Department of Psychology and Computer Science

 IHD

Abstract: Neural networks have proven effective learning machines for a variety of challenging AI tasks, as well as surprisingly good models of brain areas that underly real human intelligence. However, most successful neural networks are totally unrealistic as developmental models, because they are trained in a supervised fashion on large labelled datasets. Unsupervised approaches to learning in neural networks are thus of substantial interest for furthering artificial intelligence, both because they would enable the training of networks without the need for annotation, and because they would be better models of the kind of general-purpose learning deployed by humans. In this talk, I will describe a spectrum of recent approaches to unsupervised learning, based on ideas from cognitive science and neuroscience. First, I will discuss breakthroughs in neurally-inspired unsupervised learning of deep visual embeddings that achieve that achieve performance levels on challenging visual categorization tasks that are competitive with those of direct supervision of modern convnets. Second, I’ll discuss our work building perception systems that make accurate long-range predictions of physical futures in realistic environments, and show how these support richer self-supervised visual learning. I’ll also talk about the use of intrinsic motivation and curiosity to create interactive agents that self-curricularize, producing novel visual behaviors and learning powerful sensory representations. Finally, I’ll suggest ways in which these models are a better starting point for models of actual human visual development.

 ihd@berkeley.edu