Audio / Video

The Statistical Physics of Deep Learning

  • 01:08:53

Description

Neuronal networks have enjoyed a resurgence both in the worlds of neuroscience, where they yield mathematical frameworks for thinking about complex neural datasets, and in machine learning, where they achieve state of the art results on a variety of tasks, including machine vision, speech recognition, and language translation. Despite their empirical success, a mathematical theory of how deep neural circuits, with many layers of cascaded nonlinearities, learn and compute remains elusive. We will discuss three recent vignettes in which ideas from statistical physics can shed light on this issue. In particular, we show how dynamical criticality can help in neural learning, how the non-intuitive geometry of high dimensional error landscapes can be exploited to speed up learning, and how modern ideas from non-equilibrium statistical physics, like the Jarzynski equality, can be extended to yield powerful algorithms for modeling complex probability distributions. Time permitting, we will also discuss the relationship between neural network learning dynamics and the developmental time course of semantic concepts in infants.

Details

Title

The Statistical Physics of Deep Learning

Creator

University of California, Berkeley. Dept. of Physics

Published

Berkeley, CA, University of California, Berkeley, Dept. of Physics, October 5, 2015

Full Collection Name

Physics Colloquia

Type

Video

Format

Lecture.

Extent

1 streaming video file

Other Physical Details

digital, sd., col.

Archive

Physics Library

Note

Recorded at a colloquium held on October 5, 2015, sponsored by the Dept. of Physics, University of California, Berkeley.

originally produced as an .mts file in 2016

Speakers: Ganguli, Surya.

Usage Statement

Researchers may make free and open use of the UC Berkeley Library’s digitized public domain materials. However, some materials in our online collections may be protected by U.S. copyright law (Title 17, U.S.C.). Use or reproduction of materials protected by copyright beyond that allowed by fair use (Title 17, U.S.C. § 107) requires permission from the copyright owners. The use or reproduction of some materials may also be restricted by terms of University of California gift or purchase agreements, privacy and publicity rights, or trademark law. Responsibility for determining rights status and permissibility of any use or reproduction rests exclusively with the researcher. To learn more or make inquiries, please see our permissions policies (https://www.lib.berkeley.edu/about/permissions-policies).

Collection

Physics Colloquia

Tracks

colloquia/10-5-15Ganguli.mp4 01:08:53

Linked Resources

View record in Digital Collections.