Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning
Why Hacker News recommends it
HN commenters recommended it for years as a simple, exciting overview of the deep learning wave.
What it is
Andrew Ng presents unsupervised feature learning and deep learning, showing how algorithms can learn representations from unlabeled data instead of hand-designed features. He gives examples in vision and audio, including visualizations of features learned by successive layers.
Recommended by 9 HN commenters across 7 different months, 2011 to 2013.
What Hacker News said
Strong rec apu · 2011-05 · on “Neural Networks making a come-back?”
A good intro page with lots of links: [LINK] A lot of the newest work is under a few different names: "deep learning", "convolutional deep networks", "unsupervised feature learning", etc. Here's a great talk by Andrew Ng of Stanford, who's a recent convert to this area: [this video] (Note that this is quite a one-sided view of things, but it does convey the excitement of deep-learning researchers and the potential of what might be possible.) Although I'm not in deep learning myself (I'm a computer vision researcher), here's a TL;DR as I understand it: rather than having people in specific domains such as computer vision or speech processing create their own features, the idea is to take raw input
Strong rec brutuscat · 2012-06 · on “Microsoft Research make breakthrough in audio speech recognition”
This seems very related to this [this video] speak by Andrew Ng. It is a 40min speak, but he explains very simply how all this works for images and some examples about the audio case. It is incredible how using this deep learning techniques we can teach this "neural networks" to recognize such complicated patterns. It is like reverse engineering the brain's algorithms. BTW I took his Coursera's course about Machine Learning and it was great! I also recommend it A LOT to gather basic ML knowledge.
Strong rec kevin1024 · 2012-06 · on “Google & Stanford create a digital brain that learns to identify a human face”
Check out this (awesome) talk: [this video] There are a number of areas in which unsupervised feature learning outperforms the hand-programmed models we have been fine-tuning for decades.
Strong rec FrojoS · 2012-08 · on “Brains, Sex and Machine Learning (Geoffrey Hinton at Google Tech Talks)”
Thanks for pointing this out. Here is a similar awesome by Andrew Ng [this video] who teaches the original Coursea Machine Learning class.
Strong rec pilooch · 2012-11 · on “Scientists See Promise in Deep-Learning Programs”
the one overview I've found the most useful is [this video] (Bay Area Vision Meeting: Unsupervised Feature Learning and Deep Learning by Andrew Ng in April 2011).
Strong rec michael_nielsen · 2013-04 · on “Neural Network Visualisation in Clojure”
Andrew Ng has a fantastic example of visualizing deep belief nets: [this video] The example starts at about 18:20. You really need to have watched the previous material to understand it, but the basic idea is that he's plotting the hierarchy of features learned by successive layers of a deep neural net (a sparse autoencoder, IIRC, it's been a while since I watched it).
Recommends colincsl · 2011-06 · on “Andrew Ng: Machine Learning in Robotics”
Here is a more in-depth presentation he gave on the same subject at a Google Tech Talk: [this video] (April 11, 2011)
Recommends dave_sullivan · 2011-11 · on “Stanford computer diagnoses breast cancer more accurately than human doctor”
They kind of hint at it in the article, but sounds like automatic feature detection really helped here. That's really the future (or present): learning algos discovering better features automatically, significantly cutting down the labor involved in feature engineering. Pretty neat. Here's a video talking about current research into this at Stanford: [this video]