BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//University of California\, Berkeley//UCB Events Calendar//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
DTSTART:19701029T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:19700402T020000
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20180220T173826Z
DTSTART;TZID=America/Los_Angeles:20180228T160000
DTEND;TZID=America/Los_Angeles:20180228T170000
TRANSP:OPAQUE
SUMMARY:Algorithmic Regularization in Over-parameterized Matrix Recovery and Neural Networks with Quadratic Activations
UID:115729-ucb-events-calendar@berkeley.edu
ORGANIZER;CN="UC Berkeley Calendar Network":
LOCATION:1011 Evans Hall
DESCRIPTION:Tengyu Ma\, Facebook AI Research\n\nOver-parameterized models are widely and successfully used in deep learning\, but their workings are far from understood. In many practical scenarios\, the learned model generalizes to the test data\, even though the hypothesis class contains a model that completely overfits the training data and no regularization is applied. \n\nIn this talk\, we will show that such phenomenon occurs in over-parameterized matrix recovery models as well\, and prove that the gradient descent algorithm provides additional regularization power that prevents the overfitting. The result can be extended to learning one-hidden-layer neural networks with quadratic activations. The key insight here is that gradient descent prefers searching through the set of low complexity (that is\, low-rank) models first\, and converges to a low complexity model with a good training error if such a model exists. \n\nBased on joint work with Yuanzhi Li and Hongyang Zhang. https://arxiv.org/abs/1712.09203
URL:http://events.berkeley.edu/index.php/calendar/sn/pubaff.html?event_ID=115729&view=preview
SEQUENCE:0
CLASS:PUBLIC
CREATED:20180220T173826Z
LAST-MODIFIED:20180220T173826Z
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
X-MICROSOFT-CDO-INSTTYPE:0
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-OWNERAPPTID:-1
END:VEVENT
END:VCALENDAR