Brains, Sex, and Machine Learning

What Hacker News said

Strong rec FrojoS · 2012-11 · on “Microsoft turns spoken English into spoken Mandarin – in the same voice”

Here is a GREAT talk by Geoffrey Hinton (the Prof running said lab) [this video] where he explains the method. Unfortunately, even though it was posted three times to HN [LINK] it never made the fron page. Here is the my summary and comment: " Great talk. I don't know much about artificial neural networks (ANN) and even less about natural ones, but I have the feeling that I learnt a lot from this video. If I understand correct, Hinton uses so many artificial neurons compared to the amount of learning data, that you would usually see an overfitting effect. However, his ANN's randomly shut of a substantial part (~50%) of the neurons during each learning iteration. He calls this "dropout". Therefore

Strong rec pilooch · 2013-05 · on “Andrew Ng and the Quest for the New AI”

It seems the relationship between the brain and deep learning has evolved in such a way that the later can help with insights into how the former works. In this regard, I thought I would mention the extraordinary simple and elegant talk by G. Hinton last summer: [this video] It starts from a simple and clever improvement to an existing deep learning method and ends up with beautiful (and simple!) insights on why neurons are using simple spikes to communicate.

Strong rec eveningcoffee · 2015-11 · on “Neural Networks with Few Multiplications”

I would highly recommend this video [this video] from Geoffrey Hinton in 2012 that I think is related to this research.

Recommends psb217 · 2012-12 · on “Why use SVM”

In addition to what bravura said, Geoff Hinton's group at UT has recently introduced a new approach to training neural networks that they call "dropout". You can read about it in their paper ( [LINK] ) or watch Hinton describe it in a recent talk he gave at Google ( [this video] ). Roughly speaking, dropout training provides a strong regularizing effect through a sort of model averaging that is conceptually related to the well-known bagging approach from which random forests derive their power and flexibility. Dropout training has already produced state-of-the art results on several time-worn standard benchmarks and helped Hinton's group win a recent Kaggle competition (for an overview of their a

Recommends lightcatcher · 2013-05 · on “Unsupervised Feature Learning and Deep Learning Tutorial”

Using dropout with deep neural networks is an (extremely cheap) way to gain many of the benefits of using an ensemble. If you want a great overview of the dropout technique, watch this tech talk by Hinton: [this video] Also, the recent maxout algorithm (from Montreal group, authors of Theano) that got state of the art results on several datasets is essentially just an algorithm designed to do particularly well with dropout (as far as I understand it).

Recommends taliesinb · 2013-05 · on “Machine Learning as a Service”

Speaking of bagging: Geoff Hinton talks a bit about model averaging and has a very stimulating pair of hypotheses related to this, which are: a) genetic recombination in the form of sex might be "bagging across genes" that prevents tight co-adaptation (= overfitting on the evolutionary timescale) b) the brain uses noisy discrete firing instead of continuous communication because it allows for "bagging across network topologies". The bulk of his talk is about how dropout in neural nets can be used to accomplish some of the same performance benefits that model averaging gives on other algorithms (at a much smaller relative computational cost). Here's the talk: [this video]

Recommends carlob · 2014-02 · on “AI”

As far as I know there is a decent amount of information against the fact that brains are quantum computers. On the other hand there is a talk by Hinton on youtube [1] that sheds interesting light on some variants of deep learning and the way the brain uses (classical) noise. [1] [this video]

Cites iandanforth · 2012-10 · on “Large Scale Visual Recognition Challenge 2011 - Results”

Hinton's team (SuperVision) uses an interesting 'dropout' technique. He gave a Google Tech Talk on this back in June. [this video] And an older talk that covers some of what a deep convolutional net is: [LINK]