BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//University of California\, Berkeley//UCB Events Calendar//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
DTSTART:19701029T020000
RRULE:FREQ=YEARLY;BYMONTH=11;BYDAY=1SU
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:19700402T020000
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=2SU
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20180828T110108Z
DTSTART;TZID=America/Los_Angeles:20180830T161000
DTEND;TZID=America/Los_Angeles:20180830T170000
TRANSP:OPAQUE
SUMMARY:Mathematics Department Colloquium / Applied Math Seminar / Statistics Seminar: Stochastic Gradient Descent: Strong convergence guarantees – without parameter tuning
UID:119345-ucb-events-calendar@berkeley.edu
ORGANIZER;CN="UC Berkeley Calendar Network":
LOCATION:60 Evans Hall
DESCRIPTION:Rachel Ward\, UT Austin\n\nStochastic Gradient Descent is the basic optimization algorithm behind powerful deep learning architectures which are becoming increasingly omnipresent in society. However\, existing theoretical guarantees of convergence rely on knowing certain properties of the optimization problem such as maximal curvature and noise level which are not known a priori in practice. Thus\, in practice\, hyper parameters of the algorithm such as the stepsize are tuned by hand before training\, taking days or weeks. In this talk\, we discuss a modification of Stochastic Gradient Descent with an adaptive "on the fly" step size update known as AdaGrad which is used in practice but until now did not come with any theoretical guarantees. We provide the first such guarantees\, showing that Stochastic Gradient Descent with AdaGrad converges to a near-stationary point of a smooth loss function\, at a rate which nearly matches the "oracle" rate as if the curvature of the loss function and noise level on the stochastic gradients were known in advance. We also demonstrate its favorable empirical performance on deep learning problems compared to pre-tuned state-of-the-art algorithms.
URL:http://events.berkeley.edu/index.php/calendar/sn/pubaff.html?event_ID=119345&view=preview
SEQUENCE:0
CLASS:PUBLIC
CREATED:20180828T110108Z
LAST-MODIFIED:20180828T110309Z
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
X-MICROSOFT-CDO-INSTTYPE:0
X-MICROSOFT-CDO-IMPORTANCE:1
X-MICROSOFT-CDO-OWNERAPPTID:-1
END:VEVENT
END:VCALENDAR