Last revised 7/27/2026

[Inforthon] Machine Learning - Why Machines Can Now Do Things We Cannot Explain

Secondary

From GOFAI and the AI winters to neural networks, gradient descent, and the bias-variance tradeoff

Machine learning is the technology behind skin cancer diagnosis that rivals dermatologists, Go programs that defeat world champions, and spam filters that adapt faster than rule-writers can keep up. This two-part collection builds the complete conceptual and mathematical foundation. The first part traces the history of AI — from the Dartmouth optimism to the AI winters and the statistical turn — and explains the three ingredients (data, compute, algorithms) that made modern ML possible. The second part develops the supervised learning framework from first principles: feature vectors, the model as a parameterised function, loss functions (MSE, cross-entropy), gradient descent, and the bias-variance tradeoff.

Computer ScienceMachine LearningAICompetition PrepInforthon

2Modules13Sessions119Cards31Quizzes

Modules in this Collection’s System

Hover a module to read it directly

The Phenomenon Before the Theory

6Sessions

A New Kind of Programming

7Sessions

What You'll Walk Away With

The gradient descent update rule θ ← θ − η∇L(θ) with all symbols defined
A comparison table: supervised vs. unsupervised vs. reinforcement learning (feedback type, goal, examples)
The bias-variance tradeoff: underfitting (high bias) vs. overfitting (high variance) with the remedy for each
The three ML ingredients — data, compute, algorithms — with one key historical fact per ingredient
Cross-entropy loss formula for binary classification with the intuition for why it penalises confident wrong predictions

You'll Have Answers To

?What distinguishes machine learning from conventional programming, and why does this distinction matter?
?Why did rule-based AI (GOFAI) fail at perceptual tasks, and what is Moravec's Paradox?
?What caused the two AI winters, and what changes finally enabled the deep learning breakthrough?
?What are the three families of machine learning, and what distinguishes supervised, unsupervised, and reinforcement learning?
?How did AlexNet's 2012 ImageNet result differ from previous approaches, and why was it a turning point?
?In the supervised learning framework, what is a feature vector, a label, and a loss function?
?How does gradient descent work, and what role does the learning rate play?
?What is the difference between training loss and generalisation, and why does overfitting occur?
?What does the bias-variance tradeoff say about model complexity and generalisation performance?
?Why is cross-entropy loss preferred over MSE for classification problems with probability outputs?

Critical Concepts Explored

Tacit knowledge — expert competence that cannot be articulated as rules; what ML learns instead of encodingGOFAI (Good Old-Fashioned Artificial Intelligence) — symbolic, rule-based AI; succeeded on formal tasks, failed on perceptionMoravec's Paradox — easy for humans (perception, movement) is hard to program; hard for humans (chess) is easy to programSupervised learning — learning from labelled (input, output) pairs to generalise to new inputsFeature vector x ∈ ℝᵈ — the numerical representation of a data point; d is the number of featuresLoss function — quantifies prediction error; MSE for regression, cross-entropy for classificationGradient descent: θ ← θ − η∇L(θ) — iteratively adjusts parameters in the direction of steepest loss decreaseLearning rate η — controls step size in gradient descent; too large diverges, too small is impractically slowOverfitting — model memorises training noise; low training loss, high test loss; detected only with held-out dataBias-variance tradeoff — simple models underfit (high bias); complex models overfit (high variance)Train-test split — withhold data before training; test set evaluated exactly once to estimate generalisationLogistic regression — linear classifier using sigmoid function; decision boundary is a hyperplane in feature spaceAlexNet (2012) — deep CNN on GPUs; reduced ImageNet error from 26% to 15%; launched the deep learning eraMini-batch gradient descent — gradient computed on random subsets; cheap, regularising, widely used in practiceThe three ML ingredients — data (scale and quality), compute (GPUs, TPUs), algorithms (backprop, architectures)

Gold Quotes