Exponential family

Linear Algebra

Gaussian Discriminant Analysis

Support Vector Machine (SVM)

Regularization (mathematics)

Expectation Maximization

Unsupervised learning

Prompting

Weighted Least Squares

Generalized Linear Models

Kernel (operating system)

Neural network

Feature / Model Selection

Factor Analysis

Self-supervised Learning

Instruct Tuning

Supervised learning

Newton's Method

Probability

NumPy

Bias - Variance

Gaussian Mixture Models (GMM)

Independent Component Analysis (ICA)

In-context Learning

Multi-class Classification

Python

Backpropagation

K-means clustering

Principal Component Analysis (PCA)

Large Language Model (LLM)

Reinforcement Learning from Human Feedback (RLHF)

Logistic Regression

CS 229: Machine Learning Stanford University Winter 2023 This comprehensive course covers various machine learning principles from supervised, unsupervised to reinforcement learning. Topics also touch on neural networks, support vector machines, bias-variance tradeoffs, and many real-world applications. It requires a background in computer science, probability, multivariable calculus, and linear algebra.  This course provides a broad introduction to machine learning and statistical pattern recognition. Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs, practical advice); reinforcement learning and adaptive control. The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.  Students are expected to have the following background:

- Knowledge of basic computer science principles and skills, at a level sufficient to write a reasonably non-trivial computer program in Python/NumPy. (CS106A or CS106B, CS106X.)
- Familiarity with probability theory. (CS 109, MATH151, or STATS 116) 
- Familiarity with multivariable calculus and linear algebra (relevant classes include, but not limited to MATH 51, MATH 104, MATH 113, CS 205, CME 100.) Stanford Math 51 course text can be found [here](https://web.stanford.edu/class/math51/stanford/math51book.pdf). 

CS 229: Machine Learning

Machine Learning

Bayesian network

Message Passing

Parameter Learning

Variational Inference

Undirected Models

Sampling

Bayesian Learning

Probability theory

Exact Inference

Structured prediction

Learning Bayes Nets

MAP Inference

Structure Learning

CS 228 - Probabilistic Graphical Models Stanford University Winter 2023 An in-depth study of probabilistic graphical models, combining graph and probability theory. Equips students with the skills to design, implement, and apply these models to solve real-world problems. Discusses Bayesian networks, exact and approximate inference methods, etc. Probabilistic graphical models are a powerful framework for representing complex domains using probability distributions, with numerous applications in machine learning, computer vision, natural language processing and computational biology. Graphical models bring together graph theory and probability theory, and provide a flexible framework for modeling large collections of random variables with complex interactions. This course will provide a comprehensive survey of the topic, introducing the key formalisms and main techniques used to construct them, make predictions, and support decision-making under uncertainty.

The aim of this course is to develop the knowledge and skills necessary to design, implement and apply these models to solve real problems. The course will cover: (1) Bayesian networks, undirected graphical models and their temporal extensions; (2) exact and approximate inference methods; (3) estimation of the parameters and the structure of graphical models.  Students are expected to have background in basic probability theory, statistics, programming, algorithm design and analysis. If you are able to comfortably able to complete homework 1 then you likely have all the relevant background knowledge. ## Recommended Readings

**Corresponding Textbook**: (“PGM”) *Probabilistic Graphical Models: Principles and Techniques* by Daphne Koller and Nir Friedman. MIT Press.

**Course Notes**: Available [here](https://ermongroup.github.io/cs228-notes/). Student contributions welcome!

**Lecture Videos**: [here](https://canvas.stanford.edu/courses/166637/external_tools/3367)

**Further Readings**:

- (“GEV”) *Graphical models, exponential families, and variational inference* by Martin J. Wainwright and Michael I. Jordan. Available [online](https://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf).
- *Modeling and Reasoning with Bayesian Networks* by Adnan Darwiche. Available [online](http://ebookcentral.proquest.com/lib/stanford-ebooks/detail.action?docID=424583) (through Stanford).
- *Pattern Recognition and Machine Learning* by Chris Bishop. Available [online](https://www.microsoft.com/en-us/research/people/cmbishop/#!prml-book).
- *Machine Learning: A Probabilistic Perspective* by Kevin P. Murphy. Available [online](http://ebookcentral.proquest.com/lib/stanford-ebooks/detail.action?docID=3339490) (through Stanford).
- *Information Theory, Inference, and Learning Algorithms* by David J. C. Mackay. Available [online](http://www.inference.org.uk/mackay/itila/book.html).
- *Bayesian Reasoning and Machine Learning* by David Barber. Available [online](http://www.cs.ucl.ac.uk/staff/d.barber/brml/).

CS 228 - Probabilistic Graphical Models

Exponential family

2 courses cover this concept

CS 229: Machine Learning

CS 228 - Probabilistic Graphical Models