Machine Learning Foundations

This repo is home to the code that accompanies Jon Krohn's Machine Learning Foundations curriculum, which provides a comprehensive overview of all of the subjects — across mathematics, statistics, and computer science — that underlie contemporary machine learning approaches, including deep learning and other artificial intelligence techniques.

There are eight subjects in the curriculum, organized into four subject areas. See the "Machine Learning House" section below for detail on why these are the essential foundational subject areas:

• Linear Algebra
  • 1: Intro to Linear Algebra
  • 2: Linear Algebra II: Matrix Operations
• Calculus
  • 3: Calculus I: Limits & Derivatives
  • 4: Calculus II: Partial Derivatives & Integrals
• Probability and Statistics
  • 5: Probability & Information Theory
  • 6: Intro to Statistics
• Computer Science
  • 7: Algorithms & Data Structures
  • 8: Optimization

Later subjects build upon content from earlier subjects, so the recommended approach is to progress through the eight subjects in the order provided. That said, you're welcome to pick and choose individual subjects based on your interest or existing familiarity with the material. In particular, each of the four subject areas are fairly independent so could be approached separately.

Where and When

The eight ML Foundations subjects were initially offered by Jon Krohn as live online trainings in the O'Reilly learning platform from May-Sep 2020. To suit your preferred mode of learning, the content is now available via several channels:

• YouTube
  • All the Linear Algebra videos are live: playlist here and detailed blog post here
  • As of May 2021, new Calculus videos are being published every Wednesday: playlist here
  • Eventually, the entirety of ML Foundations content will be available on YouTube. Creating these videos to the high-quality standard you've come to expect is time-consuming and expensive, but we are getting there as quickly as we can :)
• O'Reilly
  • Linear Algebra videos published in Dec 2020 (free sample)
  • Calculus videos published in Jan 2021 (free sample)
  • Probability and Stats videos published in May 2021 (free sample)
  • Computer Science videos published in Jun 2021 (free sample)
  • Live training: The entire series is being taught online by Jon Krohn from Jul-Dec 2021 (see jonkrohn.com/talks for individual lecture dates).
• Udemy: All the Linear Algebra and Calculus content has been live in a Mathematical Foundations of ML course since Sep 2021. While this course stands alone as a complete introduction to the math subjects, Subjects 5-8 will eventually be added as free bonus material.
• Open Data Science Conference: The entire series was taught live online from Dec 2020 to Jun 2021. On-demand recordings of all these trainings are now available in the Ai+ Platform.
• Book: Chapter drafts to begin appearing in 2021

(Note that while YouTube contains 100% of the taught content, the paid options -- e.g., Udemy, O'Reilly, and ODSC -- contain comprehensive solution walk-throughs for exercises that are not available on YouTube. Some of the paid options also include exclusive, platform-specific features such as interactive testing, "cheat sheets" and the awarding of a certificate for successful course completion.)

Push Notifications

To stay informed of future live training sessions, new video releases, and book chapter releases, consider signing up for Jon Krohn's email newsletter via his homepage.

Notebooks

All code is provided within Jupyter notebooks in this directory. These notebooks are intended for use within the (free) Colab cloud environment and that is the only environment currently actively supported.

That said, if you are familiar with running Jupyter notebooks locally, you're welcome to do so (note that the library versions in this repo's Dockerfile are not necessarily current, but may provide a reasonable starting point for running Jupyter within a Docker container).

The Machine Learning House

To be an outstanding data scientist or ML engineer, it doesn't suffice to only know how to use ML algorithms via the abstract interfaces that the most popular libraries (e.g., scikit-learn, Keras) provide. To train innovative models or deploy them to run performantly in production, an in-depth appreciation of machine learning theory (pictured as the central, purple floor of the "Machine Learning House") may be helpful or essential. And, to cultivate such in-depth appreciation of ML, one must possess a working understanding of the foundational subjects.

When the foundations of the "Machine Learning House" are firm, it also makes it much easier to make the jump from general ML principles (purple floor) to specialized ML domains (the top floor, shown in gray) such as deep learning, natural language processing, machine vision, and reinforcement learning. This is because, the more specialized the application, the more likely its details for implementation are available only in academic papers or graduate-level textbooks, either of which typically assume an understanding of the foundational subjects.

The content in this series may be particularly relevant for you if:

• You use high-level software libraries to train or deploy machine learning algorithms, and would now like to understand the fundamentals underlying the abstractions, enabling you to expand your capabilities
• You’re a data scientist who would like to reinforce your understanding of the subjects at the core of your professional discipline
• You’re a software developer who would like to develop a firm foundation for the deployment of machine learning algorithms into production systems
• You’re a data analyst or A.I. enthusiast who would like to become a data scientist or data/ML engineer, and so you’re keen to deeply understand the field you’re entering from the ground up (very wise of you!)
• You're simply keen to understand the essentials of linear algebra, calculus, probability, stats, algorithms and/or data structures

The foundational subjects have largely been unchanged in recent decades and are likely to remain so for the coming decades, yet they're critical across all machine learning and data science approaches. Thus, the foundations provide a solid, career-long bedrock.

Pedagogical Approach

The purpose of this series it to provide you with a practical, functional understanding of the content covered. Context will be given for each topic, highlighting its relevance to machine learning.

As with other materials created by Jon Krohn (such as the book Deep Learning Illustrated and his 18-hour video series Deep Learning with TensorFlow, Keras, and PyTorch), the content in the series is brought to life through the combination of:

• Vivid full-color illustrations
• Paper-and-pencil comprehension exercises with fully-worked solutions
• Hundreds of straightforward examples of Python code within hands-on Jupyter notebooks (with a particular focus on the PyTorch and TensorFlow libraries)
• Resources for digging even deeper into topics that pique your curiosity

Prerequisities

Programming: All code demos will be in Python so experience with it or another object-oriented programming language would be helpful for following along with the code examples.

Mathematics: Familiarity with secondary school-level mathematics will make the class easier to follow along with. If you are comfortable dealing with quantitative information -- such as understanding charts and rearranging simple equations -- then you should be well-prepared to follow along with all of the mathematics.

Oboe

Finally, here's an illustration of Oboe, the Machine Learning Foundations mascot, created by the wonderful artist Aglaé Bassens: