Structure Learning

Bayesian network

Message Passing

Parameter Learning

Variational Inference

Undirected Models

Sampling

Bayesian Learning

Probability theory

Exact Inference

Structured prediction

Exponential family

Learning Bayes Nets

MAP Inference

CS 228 - Probabilistic Graphical Models Stanford University Winter 2023 An in-depth study of probabilistic graphical models, combining graph and probability theory. Equips students with the skills to design, implement, and apply these models to solve real-world problems. Discusses Bayesian networks, exact and approximate inference methods, etc. Probabilistic graphical models are a powerful framework for representing complex domains using probability distributions, with numerous applications in machine learning, computer vision, natural language processing and computational biology. Graphical models bring together graph theory and probability theory, and provide a flexible framework for modeling large collections of random variables with complex interactions. This course will provide a comprehensive survey of the topic, introducing the key formalisms and main techniques used to construct them, make predictions, and support decision-making under uncertainty.

The aim of this course is to develop the knowledge and skills necessary to design, implement and apply these models to solve real problems. The course will cover: (1) Bayesian networks, undirected graphical models and their temporal extensions; (2) exact and approximate inference methods; (3) estimation of the parameters and the structure of graphical models.  Students are expected to have background in basic probability theory, statistics, programming, algorithm design and analysis. If you are able to comfortably able to complete homework 1 then you likely have all the relevant background knowledge. ## Recommended Readings

**Corresponding Textbook**: (“PGM”) *Probabilistic Graphical Models: Principles and Techniques* by Daphne Koller and Nir Friedman. MIT Press.

**Course Notes**: Available [here](https://ermongroup.github.io/cs228-notes/). Student contributions welcome!

**Lecture Videos**: [here](https://canvas.stanford.edu/courses/166637/external_tools/3367)

**Further Readings**:

- (“GEV”) *Graphical models, exponential families, and variational inference* by Martin J. Wainwright and Michael I. Jordan. Available [online](https://www.eecs.berkeley.edu/~wainwrig/Papers/WaiJor08_FTML.pdf).
- *Modeling and Reasoning with Bayesian Networks* by Adnan Darwiche. Available [online](http://ebookcentral.proquest.com/lib/stanford-ebooks/detail.action?docID=424583) (through Stanford).
- *Pattern Recognition and Machine Learning* by Chris Bishop. Available [online](https://www.microsoft.com/en-us/research/people/cmbishop/#!prml-book).
- *Machine Learning: A Probabilistic Perspective* by Kevin P. Murphy. Available [online](http://ebookcentral.proquest.com/lib/stanford-ebooks/detail.action?docID=3339490) (through Stanford).
- *Information Theory, Inference, and Learning Algorithms* by David J. C. Mackay. Available [online](http://www.inference.org.uk/mackay/itila/book.html).
- *Bayesian Reasoning and Machine Learning* by David Barber. Available [online](http://www.cs.ucl.ac.uk/staff/d.barber/brml/).

CS 228 - Probabilistic Graphical Models

Machine Learning

Probability

Policy Search

Beliefs

Multiagent Reasoning

Inference

Exact Solutions

Actor Critic

Offline Belief State Planning

Online Planning

Model-Based and Model-Free Reinforcement Learning

Controller Abstractions

Simple Decisions

Policy Gradient Estimation and Optimization

Exact Belief State Planning

Policy Validation, Implications, and Summary

Representations

Bayesian Networks Representation and Inference

Approximate Value Functions

Exploration / Exploitation

Online Belief State Planning

AA 228 / CS 238 Decision Making under Uncertainty Stanford University Winter 2023 The course introduces decision making under uncertainty from a computational perspective, covering dynamic programming, reinforcement learning, and more. Prerequisites include basic probability and fluency in a high-level programming language. This course introduces decision making under uncertainty from a computational perspective and provides an overview of the necessary tools for building autonomous and decision-support systems. Following an introduction to probabilistic models and decision theory, the course will cover computational methods for solving decision problems with stochastic dynamics, model uncertainty, and imperfect state information. Topics include Bayesian networks, influence diagrams, dynamic programming, reinforcement learning, and partially observable Markov decision processes. Applications cover air traffic control, aviation surveillance systems, autonomous vehicles, and robotic planetary exploration. - You will gain a broad fundamental understanding of the mathematical models and solution methods for decision making (exercises, three quizzes).
- You will be able to implement and extend key algorithms for learning and decision making (two programming projects).
- You will be able to identify an application of the theory in this course and formulate it mathematically (proposal).
- You will gain a deep understanding of an area of particular interest and apply it to a problem (final project).
- You will be able to critique approaches to solving decision problems (peer review). basic probability and fluency in a high-level programming language. This year we will be using the textbook titled *Algorithms for Decision Making*, MIT Press, 2022. Printed copies are available for purchase or you may download it for free as a PDF. Details are [here](https://algorithmsbook.com/decisionmaking).

The textbook serves as the official lecture notes.

AA 228 / CS 238 Decision Making under Uncertainty

Artificial Intelligence

Structure Learning

2 courses cover this concept

CS 228 - Probabilistic Graphical Models

AA 228 / CS 238 Decision Making under Uncertainty