Overview
Optimisation is the part of mathematics that answers “what is the best you can do, given what you are allowed to do”. The course takes it in order: first the shape of the problem, then the direction to move in, then how far to step, then what changes when constraints close in around you.
Nothing is imported. Gradient descent, Newton's method, backtracking line search, stochastic gradient descent and the simplex algorithm are all written out, run on real data, and compared — including the times they behave badly, which is where the learning actually is.
What I built
- Gradient descent and Newton's method on the same problem, counting the iterations each one needs: 32,274 against a handful.
- Backtracking line search with the Wolfe conditions, so the step size stops being a guess.
- Stochastic gradient descent with mini-batching on 10,000 height and weight records, checked against the closed-form minimiser.
- Constrained optimisation: feasible regions, active sets and the KKT conditions.
- The simplex method, pivot by pivot, from a starting corner to the optimum.
The seven assignments
Read-only · outputs from the assignment notebooks
Each one starts from the mathematics and ends in code that has to work. Open a row to read what it was about and run the cells that produced the figures.