NTR Webinar: Surprising Properties of Loss Function in Deep Learning

NTR organizes and hosts scientific webinars on neural networks and invites speakers from all over the world to present their recent work at the webinars.

On April 13 Dmitry Vetrov, National Research University Higher School of Economics, Moscow, Russia, led a technical Zoom webinar on Surprising Properties of Loss Function in Deep Learning. 

About the webinar: 

During last several years several surprising effects have been discovered in deep neural networks. Among them are mode connectivity, double descent, gap between theory and practice in generalization, and minefields in loss landscape. 

During the talk we reviewed those effects and presented a hypothesis that can at least partly explain what’s going on during the training process and what are the properties of the loss function in DNNs. 

Based on this hypothesis we formulated experiments that provide arguments in its favor and share the results of preliminary experiments. 

Better understanding of training dynamics may lead to the development of more efficient methods for training of deep neural networks and the directions for improving existing algorithms.

Materials available:

Webinar presentation

Moderator and contact:

NTR CEO Nick Mikhailovsky: nickm@ntrlab.com.

Leave a Reply

Your email address will not be published. Required fields are marked *