Chapter 17: Learning Theory & Advanced ML
Chapter Overview
Generalization theory · Online learning & bandits · Reinforcement learning (the foundation for RLHF) · Causal inference · Sparse & factorization methods · Kernel methods
Sections
1
Learning Theory Foundations
PAC learning, VC dimension, generalization bounds
2
Online Learning & Bandits
Exploration vs exploitation, Thompson sampling, UCB
3
Reinforcement Learning
MDPs, Q-learning, policy gradient, PPO, RLHF
4
Causal Inference
SCMs, do-calculus, backdoor criterion, counterfactuals
5
Sparse & Kernel Methods
6
Nice to Know
Meta-learning, convergence, optimal transport, and more