Title: Recent Development in Neural Networks
Abstract:
Deep Learning is a set of algorithms that model data by constructing layers of neurons that represent different levels of abstraction. These methods have been applied successfully in fields such as computer vision, speech recognition, natural language processing, signal recognition, and in some cases outperform other methods by a comfortable margin.
Deep Learning started with the theory of Neural Networks. They were developed last century as a "brain inspired" way to do classification and had some success although they eventually lost popularity in the 90's. They were considered hard to train, especially if using more than 3 layers of neurons, and were often outperformed by Support Vector Machines, and even simple Linear Classifiers.
The field was however revolutionized in 2006, when Hilton et al. proposed a way to pre-train each of the layers individually in an unsupervised manner, before running a supervised gradient descent. Since then the field has taken off, under the name of Deep Learning, and is a very active area of research.
I will give an overview of advances since 2006, and especially try to shed some light on how neural networks are pre-trained, which is a delicate task. I will also give some examples of where deep learning has been applied, hopefully including Google's and Baidu's image search, and Microsoft's Speech recognition.
The talk will be mathematically simple, although it will be helpful if you are familiar with Machine Learning concepts. It will partially be based on work I did in the summer of 2013, during my internship at the Bosch Research and Technology Center in Palo Alto.