Generalization

Histories of AI

Societal impacts of AI

Vector (mathematics and physics)

Dot product

Geometric interpretations

Taking gradients

Discrete random variables

Probability distributions

Mean

Variance

Marginal distributions

Conditional distribution

Big-O notation

Computational complexity

Recurrence relation

Dynamic programming

Continuous optimization

Objective functions

Gradient descent

Machine learning

Loss minimization

Hinge loss

Equitable performance

Design and organization of features

Computation graphs

Deep learning models composition

Cross-validation (statistics)

Search problem

Breadth-first search (BFS)

Uniform cost search (UCS)

UCS heuristics

Markov Decision Process (MDP)

Transportation problem

Discounting factor

Reinforcement learning (RL)

Model-free Monte Carlo

Q-learning

Function approximation

Game theory

Minimax algorithm

Temporal difference (TD) learning

Variable-based models

Backtracking search

Gibbs sampling

Bayesian network

Laplace smoothing

Logic

Entailment

Satisfiability

Soundness

Modus ponens

First-order logic

Unification (computer science)

Python

Linear regression

Linear classification

Stochastic gradient descent (SGD)

Non-linear functions

Neural network

Backpropagation

K-means clustering

Exhaustive search

Depth-first search (DFS)

UCS correctness

Relaxed search problems

Dice game

Policy evaluation

Value iteration

Model-based Monte Carlo

State–action–reward–state–action (SARSA)

Epsilon-greedy exploration

Deep Reinforcement Learning

Halving game

Alpha-beta pruning

Nash equilibrium

Factor graphs

AC-3 algorithm

Markov random field

Hidden Markov Model (HMM)

Maximum Likelihood Estimation (MLE)

Propositional calculus (Propositional logic)

Contradiction

Model checking

Completeness

Horn clauses

Substitution (logic)

Skolem functions

CS 221 Artificial Intelligence: Principles and Techniques Stanford University Autumn 2022-2023 Stanford's CS 221 course teaches foundational principles and practical implementation of AI systems. It covers machine learning, game playing, constraint satisfaction, graphical models, and logic. A rigorous course requiring solid foundational skills in programming, math, and probability. The goal of artificial intelligence (AI) is to tackle complex real-world problems with rigorous mathematical tools. In this course, you will learn the foundational principles and practice implementing various AI systems. Specific topics include machine learning, search, Markov decision processes, game playing, constraint satisfaction, graphical models, and logic.  This course is fast-paced and covers a lot of ground, so it is important that you have a solid foundation in a number of areas. Here are the basic skills that you need and the classes that teach those skills:

- Programming (ideally Python): [CS 106A](http://www.stanford.edu/class/cs106a/), [CS 106B](http://www.stanford.edu/class/cs106b/), [CS 107](http://www.stanford.edu/class/cs107/)
- Discrete math, mathematical rigor: [CS 103](http://www.stanford.edu/class/cs103/)
- Probability: [CS 109](http://www.stanford.edu/class/cs109/)
- Linear algebra: [Math 51](https://web.stanford.edu/class/math51/textbook.html)

It is less important that you know particular things (e.g., we don't use eigenvectors in this course even though that's a pillar of any linear algebra course), and more important that you've done enough related things that you feel at ease with it. While it is possible to fill in the gaps, this course does move quickly, and ideally you want to be focusing your energy on learning AI rather than catching up on prerequisites. We have made a few [prerequisite modules](https://stanford-cs221.github.io/autumn2022/modules) that you can review to refresh your memory, and the first homework (foundations) will allow you to also get some practice on these basics. ### Further Reading

There are no required textbooks for this class, and you should be able to learn everything from the lecture notes and homeworks. However, if you would like to pursue more advanced topics or get another perspective on the same material, here are some great resources:

- [Russell and Norvig. Artificial Intelligence: A Modern Approach](http://aima.cs.berkeley.edu/). A comprehensive reference for all the AI topics that we will cover.
- [Koller and Friedman. Probabilistic Graphical Models](http://mitpress.mit.edu/books/probabilistic-graphical-models). Covers factor graphs and Bayesian networks (this is the textbook for CS228).
- [Sutton and Barto. Reinforcement Learning: An Introduction](https://mitpress.mit.edu/books/reinforcement-learning). Covers Markov decision processes and reinforcement learning (free online).
- [Hastie, Tibshirani, and Friedman. The Elements of Statistical Learning](https://web.stanford.edu/~hastie/ElemStatLearn/). Covers machine learning from a rigorous statistical perspective (free online).
- [Tsang. Foundations of Constraint Satisfaction](http://www.bracil.net/edward/fcs.html). Covers constraint satisfaction problems (free online).

Note that some of these books use different notation and terminology from this course, so it may take some effort to make the appropriate connections.

CS 221 Artificial Intelligence: Principles and Techniques

Artificial Intelligence

Property-preserving lossy compression

Nearest Neighbor

k-d tree

Regularization (mathematics)

L1 regularization

Matrix compression

Laplacian of a graph

Reservoir sampling

Markov chain Monte Carlo (MCMC)

Majority elements

Distance-preserving compression

PAC Guarantees

Linear-Algebraic Techniques

De-noising

Eigenvectors/eigenvalues

Markov's inequality

The Fourier Perspective

Linear programming

Dimensionality Reduction

Approximate heavy hitters

Locality-Sensitive Hashing (LSH)

MinHash

Principal Component Analysis (PCA)

Matrix completion

Spectral embeddings

Chebyshev‘s inequality

Convolution

Convex optimization

Empirical risk minimization

Bloom filters

Jaccard Similarity

Polynomial embedding

Maximizing variance

Uniqueness of low-rank tensor factorizations

Graph coloring

Importance Sampling

Sparse Vector/Matrix Recovery

Privacy Preserving Computation

Count-min sketch

Euclidean Distance

Johnson-Lindenstrauss Transform

Random projection

Power iteration algorithm

Jenrich's algorithm

Spectral clustering

Markov Chains

Compressed Sensing

Differential privacy

Similarity Search

Lp distance

L2 regularization

Singular Value Decomposition (SVD)

Spectral Graph Theory

Interpretations of the second eigenvalue

Stationary distributions

Mathematical Programming

CS 168: The Modern Algorithmic Toolbox Stanford University Spring 2022 CS 168 provides a comprehensive introduction to modern algorithm concepts, covering hashing, dimension reduction, programming, gradient descent, and regression. It emphasizes both theoretical understanding and practical application, with each topic complemented by a mini-project. It's suitable for those who have taken CS107 and CS161.  This course will provide a rigorous and hands-on introduction to the central ideas and algorithms that constitute the core of the modern algorithms toolkit. Emphasis will be on understanding the high-level theoretical intuitions and principles underlying the algorithms we discuss, as well as developing a concrete understanding of when and how to implement and apply the algorithms. The course will be structured as a sequence of one-week investigations; each week will introduce one algorithmic idea, and discuss the motivation, theoretical underpinning, and practical applications of that algorithmic idea. Each topic will be accompanied by a mini-project in which students will be guided through a practical application of the ideas of the week. Topics include modern techniques in hashing, dimension reduction, linear and convex programming, gradient descent and regression, sampling and estimation, compressive sensing, and linear-algebraic techniques (principal components analysis, singular value decomposition, spectral techniques).  CS107 and CS161, or permission from the instructor. 

CS 168: The Modern Algorithmic Toolbox

Theoretical Computer Science

Generalization

2 courses cover this concept

CS 221 Artificial Intelligence: Principles and Techniques

CS 168: The Modern Algorithmic Toolbox