Machine Learning

Author

Phil Chodrow

Published

January 15, 2024

Welcome

This is a set of notes developed for an undergraduate course in machine learning. The target audience for these notes are undergraduates in computer science who have completed first courses in linear algebra and discrete mathematics. These notes draw on many sources, but are somewhat distinctive in the following ways:

  • The technical focus is almost exclusively on smooth methods for empirical risk minimization in supervised regression and, especially, classification. These notes do not attempt to be a broad survey of all of machine learning. Especially important topics which are largely untreated include decision trees, random forests, and most unsupervised techniques.
  • The social impacts of automated decision technologies–including bias, fairness, and harm–are considered as first-class topics which occupy a substantial fraction of the notes.
  • Minimal familiarity with probability is assumed. Continuous probability does not appear explicitly, and discrete probability is introduced “from the ground up” as needed.

Pedagogical Features

These notes are explicitly designed for undergraduate instruction in computer science. For this reason:

  • Computational examples are integrated into the text and shown throughout.
  • Live versions of lecture notes are supplied as downloadable Jupyter Notebooks, with certain code components removed. The purpose is to facilitate live-coding in lectures.

Use and Reuse

These notes were written by Phil Chodrow for the course CSCI 0451: Machine Learning at Middlebury College. All are welcome to use them for educational purposes. Attribution is appreciated but not expected.

Source Texts

These notes draw on several source texts, most of which are available for free online. These are:

  • Hardt and Recht (2022) is the primary influence for the overall arc of the notes.
  • A Course in Machine Learning by Hal Daumé III is an accessible introduction to many of the topics and serves as a useful source of supplementary readings.

Additional useful readings:

  • Abu-Mostafa, Magdon-Ismail, and Lin (2012): Learning From Data: A Short Course
  • Barocas, Hardt, and Narayanan (2023) is an advanced text on questions of fairness in automated decision-making for readers who have some background in probability theory.
  • Bishop (2006) and Murphy (2022) are advanced texts which are most suitable for advanced readers who have already taken at least one course in probability theory.
  • Deisenroth, Faisal, and Ong (2020) and Kroese et al. (2020) are useful readings focusing on some of the mathematical fundamentals.
  • Zhang, Lipton, and Li (2023) tells a helpful story of the fundamentals of deep learning.

Acknowledgements

This site was generated using the Quarto publishing system. It is hosted on GitHub and published via GitHub Pages.

References

Abu-Mostafa, Yaser S., Malik Magdon-Ismail, and Hsuan-Tien Lin. 2012. Learning from Data: A Short Course. S.l. https://amlbook.com/.
Barocas, Solon, Moritz Hardt, and Arvind Narayanan. 2023. Fairness and Machine Learning: Limitations and Opportunities. Cambridge, Massachusetts: The MIT Press. https://fairmlbook.org/pdf/fairmlbook.pdf.
Bishop, Christopher M. 2006. Pattern Recognition and Machine Learning. Information Science and Statistics. New York: Springer. https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf.
Deisenroth, Marc Peter, A. Aldo Faisal, and Cheng Soon Ong. 2020. Mathematics for Machine Learning. Cambridge, UK New York, NY: Cambridge University Press. https://mml-book.github.io/book/mml-book.pdf.
Hardt, Moritz, and Benjamin Recht. 2022. Patterns, Predictions, and Actions. Princeton University Press. https://mlstory.org/pdf/patterns.pdf.
Kroese, Dirk P., Zdravko I. Botev, Thomas Taimre, and Radislav Vaisman. 2020. Data Science and Machine Learning: Mathematical and Statistical Methods. Chapman & Hall/CRC Machine Learning & Pattern Recognition Series. Boca Raton London New York: CRC Press, Taylor & Francis Group.
Murphy, Kevin P. 2022. Probabilistic Machine Learning: An Introduction. Adaptive Computation and Machine Learning Series. Cambridge, Massachusetts: The MIT Press. https://probml.github.io/pml-book/book1.html.
Zhang, Aston, Zachary Lipton, and Mu Li. 2023. Dive into Deep Learning. Cambridge, UK: Cambridge University Press.



© Phil Chodrow, 2025