Overwhelmed by Machine Learning - is there an ML101 book?It seems like there are so many subfields linked to Machine Learning. Is there a book or a blog that gives an overview of those different fields and what each of them do, maybe how to get started, and what background knowledge is required?
ACCEPTED]
Here's the best description I've ever heard of Machine Learning:
Machine learning is actually a software method. It's a way to generate software. So, it uses statistics but it's fundamentally... it's almost like a compiler. You use data to produce programs. - John Platt, Distinguished Scientist at Microsoft Research [1] in his Future of AI series talk (2:17:53) [2]
Some even argue that " everything that algorithms was to computer science 15 years ago, machine learning is today [3]."
For more details, I'd recommend starting out with a fun intro to what's possible such as Peter Norvig's Theorizing from Data [4] talk, a peek at what DeepMind is doing [5], or more recently the Future of AI series of talks [6] (that I quoted from above).
Next get your hands dirty with Jeremy Howard's " Getting In Shape For The Sport of Data Science [7]." It's a great pragmatic overview of actually working with data.
Once you've played around a bit, watch Ben Hamner's " Machine Learning Gremlins [8]" for a nice pragmatic disclaimer of what can easily go wrong when doing machine learning.
I wrote a blog post " Computing Your Skill [9]" after spending months trying to understand TrueSkill [10], the ML system that does matchmaking and ranking on Xbox Live. The post goes into some foundational statistics needed for further study in machine learning.
Perhaps the best way to learn is to just try it. One approach is to try a Kaggle [11] competition that sounds interesting to you. Even though I don't do great on the leaderboards there, I always learn things when I try a competition.
After that you've done the above, I'd then recommend something more formal like Andrew Ng's online class [12]. It's at the college level, but approachable. If you've done all the above steps, you'll be more motivated to not give up when you hit some harder things.
As you continue, you'll learn about things such as R [13] and its many packages [14], SciPy [15], Cross Validation [16], Bayesian thinking [17], Deep [18] Learning [19], and much [20] much more [21].
DISCLAIMER: I work at Kaggle and several of the above links mention Kaggle, but I believe they're a fantastic place to start.
[1] http://research.microsoft.com/en-us/people/jplatt/videolectures.net has a large collection of Machine Learning videos [1] . One very good technical introductory lecture on the site is Machine Learning, Probability and Graphical Models [2] by Sam Roweis.
A good overview of the field is Tom Mitchell's seminar The Discipline and Future of Machine Learning [3]. Here is a direct link to the video [mov] [4]. And the Syllabus [5] page has a good list of recommended texts:
Ethem Alpaydin's Introduction to Machine Learning [1] is a pretty accessible overview of the field.
If you're feeling overwhelmed by the other options you might want to start with it first.
[1] https://rads.stackoverflow.com/amzn/click/com/0262012111Two of the best textbooks out there are:
Pattern Classification [1] by Duda and Hart, and
Pattern Recognition and Machine Learning [2] by Bishop.
Another good resource is MIT's Open CourseWare site for their Machine Learning class [3].
[1] https://rads.stackoverflow.com/amzn/click/com/0471056693I really like the Machine Learning course on Coursera [1]. I find the short lectures very easy to digest.
[1] https://class.coursera.org/ml/lecture/preview/indexI found " Programming Collective Intelligence [1]" to be the book that really helped me (with practical examples) and an "Algorithm Beastiary" at the end.
[1] https://rads.stackoverflow.com/amzn/click/com/0596529325Dr Yaser Abu Mustafa's Intro course is also in detailed and he explained it quite well
http://work.caltech.edu/telecourse.html
Artificial Intelligence: A Modern Approach [1] is the most common text book for introductory AI courses.
Witten and Frank's book on Data Mining [2] is a little easier to digest if that topic is what appeals to you.
[1] https://rads.stackoverflow.com/amzn/click/com/0137903952You are right to feel that there are lots of sub-fields to ML.
Machine Learning in general is basically just the idea of Algorithms which improve over time. If you're simply curious, some random topics that come to mind include:
Classification, Association analysis, Clustering, Decision Trees, Genetic Algorithms, Concept Learning
As far as books go:
I'm currently using Introduction to Data Mining [1] for a course right now. It covers quite a few of the topics I've listed above and usually has examples of algorithms/uses in each section.
You don't need too much background knowledge to understand a lot of the topics. Most algorithms have some math underlying them which is used to improve the results, and you obviously need to be comfortable with general programming/data structures.
[1] https://rads.stackoverflow.com/amzn/click/com/0321321367i'd recommand you take a look at ml-class.org.
I've been using 'Machine Learning: An algorithmic Perspective' by Stephen Marsland. And I think the approach is awesome. The author has put up the python code on his site. So you can actually download the code and look at it just to take a peek at how things work.
http://www-ist.massey.ac.nz/smarsland/MLbook.html
Try A First Encounter with Machine Learning [1], it's a freely available course for undergraduate level.
[1] http://www.ics.uci.edu/~welling/teaching/273ASpring10/IntroMLBook.pdfThe Machine Learning subreddit [1] has interesting links for all levels.
[1] http://www.reddit.com/r/MachineLearning/