The Statistical Physics of Deep Learning
- 01:08:53
Description |
---|
Neuronal networks have enjoyed a resurgence both in the worlds of neuroscience, where they yield mathematical frameworks for thinking about complex neural datasets, and in machine learning, where they achieve state of the art results on a variety of tasks, including machine vision, speech recognition, and language translation. Despite their empirical success, a mathematical theory of how deep neural circuits, with many layers of cascaded nonlinearities, learn and compute remains elusive. We will discuss three recent vignettes in which ideas from statistical physics can shed light on this issue. In particular, we show how dynamical criticality can help in neural learning, how the non-intuitive geometry of high dimensional error landscapes can be exploited to speed up learning, and how modern ideas from non-equilibrium statistical physics, like the Jarzynski equality, can be extended to yield powerful algorithms for modeling complex probability distributions. Time permitting, we will also discuss the relationship between neural network learning dynamics and the developmental time course of semantic concepts in infants. |
Details |
|
---|---|
Title |
The Statistical Physics of Deep Learning |
Creator |
University of California, Berkeley. Dept. of Physics |
Published |
Berkeley, CA, University of California, Berkeley, Dept. of Physics, October 5, 2015 |
Full Collection Name |
Physics Colloquia |
Type |
Video |
Format |
Lecture. |
Extent |
1 streaming video file |
Other Physical Details |
digital, sd., col. |
Archive |
Physics Library |
Note |
Recorded at a colloquium held on October 5, 2015, sponsored by the Dept. of Physics, University of California, Berkeley. originally produced as an .mts file in 2016 Speakers: Ganguli, Surya. |
Usage Statement |
Researchers may make free and open use of the UC Berkeley Library’s digitized public domain materials. However, some materials in our online collections may be protected by U.S. copyright law (Title 17, U.S.C.). Use or reproduction of materials protected by copyright beyond that allowed by fair use (Title 17, U.S.C. § 107) requires permission from the copyright owners. The use or reproduction of some materials may also be restricted by terms of University of California gift or purchase agreements, privacy and publicity rights, or trademark law. Responsibility for determining rights status and permissibility of any use or reproduction rests exclusively with the researcher. To learn more or make inquiries, please see our permissions policies (https://www.lib.berkeley.edu/about/permissions-policies). |
Collection |
Physics Colloquia |
Tracks |
colloquia/10-5-15Ganguli.mp4 01:08:53 |
Linked Resources |