Intermediate ML

This is a work-in-progress draft of intermediate-level (B2) machine learning materials. Thanks to LLMs for the high quality; any errors are mine.

Formalisms

• Probability
• Conditional expectation
• Risk minimization
• Optimization
• MLE and MAP estimation
• Regularization
• Exponential family
• Linear algebra
• Information theory
• Singular value decomposition (SVD)

Linear models

• Ordinary least squares (OLS)
• Weighted least squares (WLS)
• Partial least squares (PLS)
• Generalized linear models (GLM)
• Generalized additive models (GAM)

Non-linear models

• Nonlinear least squares
• Support vector machine (SVM)
• Support vector regression (SVR)

Ensemble methods

• Decision trees
• Bagging and stacking
• Classical boosting
• Gradient boosting

Exploratory methods

• Principal component analysis (PCA)
• Density estimation
• Clustering