4 Classification

Categorical or qualitative outcome variables are ubiquitous. We review some supervised learning methods for classification, and see how these may be applied to observational causal inference.

Materials

Link	Type	Description
html pdf	Slides	Classification and logistic regression
[html] Rmd	Notebook	Classification, class balance, ROC curves
pdf html Rmd	Notebook	Gradient descent and numeric differentiation
html	Exercises	First exercise set

To be updated

Preparation

Required reading

• ISLR Chapter 4. This chapter is a bit lengthy, so it’s OK if you don’t carefully follow all the mathematics in sections 4.4 and 4.5 as long as you understand the concepts.
• MLstory Start Chapter 5 on optimization, read the first two sections on Optimization Basics and on Gradient Descent. You can stop when you reach Proposition 3.

Supplemental reading

• Wikipedia on Newton’s method and gradient descent (good for pictures and animations)
• CASI Chapter 4 for MLE theory, Chapter 8 sections 8.1-8.3 (more advanced material)

Carving nature at its joints

“A good cook gets a new knife every year; he chops! Mediocre cooks change knives monthly; they hack. My knife now has 19 years on it; it’s carved several thousand oxen and the edge is as if I had just taken it from the sharpener. Those joints have gaps, and the knife’s edge no thickness, to put something infinitesimally thin in an empty space?! Effortless! It even allows the edge wander in with ample room to play. That is why, with 19 years on it, this knife’s edge is grindstone fresh.” - Butcher Ding, the Zhuangzi