Brett Walenz

This week is the last week of the Stanford Machine Learning course (ml-class.org), so here is my basic review of the course. Overall, the quality is quite good. The lectures are easily the most valuable part, as Andrew Ng is very organized and provides insightful explanations over the approaches and algorithms.

The homework problems use Octave, a mostly clone version of Matlab (including down to language spec), so it’s pretty easy to pick up for most math-inclined people.

The homework assignments themselves were mildly… disappointing? I’m not sure that’s the right word, but I feel they were really lacking in improving understanding of the primary concept at hand. At the start of the course, the homework assignments are concerned with implementing cost functions for linear regression and neural networks. At the end of the course, the homework assignments are concerned with implementing cost functions for k-means clustering – so it gets fairly repetitive, and doesn’t really make you aware of the steps involved with the different algorithms. Most of the supporting code that goes into the algorithms is written for you by the course team. Let’s take a look at what I mean.

In the following example, the course walks you through a collaborative filtering example to recommend movies that you might enjoy based on previous ratings. There are a number of steps that go into it:

1. Load the data
2. Normalize the ratings
3. Set initial values for X and Theta
4. Create a cost function
5. Find values of Theta that minimize the cost function
6. Use X and Theta to produce a prediction

In this assignment, the only step you do is 4. Creating a cost function, so you don’t really get a sense of what is going on unless you step through the code yourself, and it generally moves fast enough that I’m guessing most people don’t. And, for the most part, that’s my only criticism. The course does talk about, from week to week, mean normalization, regularization, etc., but you never really ‘connect the dots’ in an assignment. It’s always:

• Fill out formula
• Run magic program
• Did formula look right? If so, submit.
• If not, look at formula again.

Gripe aside, again, I really enjoyed the class. The lectures really are outstanding. And he pays a lot of attention at the start to talk about regression (linear and logistic), and neural networks – while explaining them in a clear and concise manner, avoiding a lot of ugly statistics that most people get lost in. Actually, that’s another nice part. He does a really good job of mixing data mining terms and statistics terms so that you understand there are multiple ways to describe something. That way if you go someplace else to read about it, you won’t be lost(weights vs. parameters, fitting vs. learning).

The other topics covered are SVM’s, K-means clustering, anomaly detection, and practical applications of machine learning, as well as some large scale discussions.

So I’d recommend it, with one caveat – make sure you don’t fall into the trap of “make formula go”, and understand the underlying connecting parts.