Lecture 2 | Machine Learning (Stanford)
Why Stack Overflow recommends it
Stack Overflow users link it as the source for the gradient descent and linear regression algorithm they are implementing or asking about, especially the update rule shown around the 25 minute mark.
Recommended by 1 people across 1 different months, 2011 to 2015.
What it is
Lecture 2 of Stanford's CS 229 Machine Learning course, taught by Andrew Ng in 2008. It covers linear regression, gradient descent and the normal equations as the first concrete supervised learning algorithms.
What people said
Recommends mpenkov · 2013-04 · Stack Overflow · on “Can someone give me an overview of gradient descent?”
Gradient descent is a method for finding the minimum/maximum value of some multidimensional function. To keep things simple, imagine looking for the peak of a hill. In this scenario, we're looking for a maximum (altitude) in 3 dimensions (longitude, latitude, altitude). The function is the surface of the hill, with two inputs (longitude, latitude) and one output (altitude). If you had to use gradient descent, you'd do this: Work out the slope in each direction at your current location Move a small distance in the direction of the steepest positive incline If reached convergence, then stop. Else, go back to step 1. Convergence means that the result is not going to change significantly if you
Cites howard · 2011-10 · Stack Overflow · on “Gradient descent algorithm won't converge”
I'm trying to write out a bit of code for the gradient descent algorithm explained in the Stanford Machine Learning lecture ( lecture 2 at around 25:00 [this video] ). Below is the implementation I used at first, and I think it's properly copied over from the lecture, but it doesn't converge when I add large numbers ( >8 ) to the training set. I'm inputting a number X , and the point (X,X) is added to the training set, so at the moment, I'm only trying to get it to converge to y=ax+b where a=1=theta\[1\] and b=0=theta\[0\] . The training set is the array x and y , where (x[i],y[i]) is a point. void train() { double delta; for (int i = 0; i < x.size(); i++) { delta = y[i]-hypothesis(x[i]); theta[1
Cites Davors72 · 2015-05 · Stack Overflow · on “Linear regression poor gradient descent performance”
The algorithm is taken from the Stanford machine learning course lecture link [this video] , as well as a few other videos, thought it is pretty standard. The cost grows after each iteration only when the learning rate is too high (which I think is wrong), if the learning rate is lower it will *slowly* reduce.