5 Optimization and model complexity

Machine learning is broadly about estimating functions using optimization algorithms. We can think of these as searching through a space of functions to find one that minimizes a measure of inaccuracy or loss.

Joshua Loftus
10-07-2021

Materials

Link Type Description
html pdf Slides Optimization and model complexity
html Notebook Gradient descent
Rmd Notebook Stochastic gradient descent
Rmd Notebook Stepwise variable selection

To be updated

Preparation

Required reading

Supplemental reading

Optimization

Overfitting

Slides, notebooks, exercises

Slides for optimization video (PDF)

Slides for overfitting video (PDF)

Notebook for generalization (partially complete)

Notebook for optimization (partially complete)

Notebook for regularization (partially complete)

Exercises

Reuse

Text and figures are licensed under Creative Commons Attribution CC BY 4.0. The figures that have been reused from other sources don't fall under this license and can be recognized by a note in their caption: "Figure from ...".

Citation

For attribution, please cite this work as

Loftus (2021, Oct. 7). machine learning 4 data science: 5 Optimization and model complexity. Retrieved from http://ml4ds.com/weeks/05-optimization/

BibTeX citation

@misc{loftus20215,
  author = {Loftus, Joshua},
  title = {machine learning 4 data science: 5 Optimization and model complexity},
  url = {http://ml4ds.com/weeks/05-optimization/},
  year = {2021}
}