Online Learning

Stochastic Optimization

Core-sets and importance sampling

VC dimension

Inverse Problems

Stochastic gradient descent (SGD)

Noncommutative Chernoff bounds

Rademacher complexity

Learning representations

Graph

Metric embedding

Kaczmarz algorithm

Matrix approximation

Restricted isometries

Randomized algorithm

Diagonally dominant systems

Compressed Sensing

Random Matrices

Covering numbers

Statistical complexity

CS 294 - The Mathematics of Information and Data UC Berkeley Fall 2013 This course investigates the mathematical principles behind data and information analysis. It brings together concepts from statistics, optimization, and computer science, with a focus on large deviation inequalities, and convex analysis. It's tailored towards advanced graduate students who wish to incorporate these theories into their research. This course will explore the foundations of an emerging discipline: the mathematics of information and data. Through recent and classic texts in mathematical statistics, optimization, and computer science, we will find unifying themes in these three disciplinary approaches. We will draw connections between how we analyze running time, statistical accuracy, and implementation of data-driven computations. We will focus in particular on large deviation inequalities, convex analysis and their applications in minimax statistics; sparse and stochastic optimization; and discrete and convex geometry. This course is ideal for advanced graduate students who would like to apply these theoretical and algorithmic developments to their own research. The current list of topics (which will change depending on the course we chart) is:

1. Stochastic Optimization
    - stochastic gradients, online learning, and the Kaczmarz algorithm
    - core sets and importance sampling
    - randomized algorithms for linear systems
1. Random Matrices
    - Elementary analysis of random matrices
    - Graph sparsification, frames, and matrix approximation
    - Noncommutative Chernoff Bounds
1. Average Case Analysis of Optimization Problems
    - covering numbers, VC dimension, rademacher complexity
    - metric embedding and restricted isometries
    - compressed sensing and all that it has wrought Consent of the instructor is required. Graduate level courses in probability and optimization will be necessary. 

CS 294 - The Mathematics of Information and Data

Theoretical Computer Science

Decision Trees

Clustering

PAC Learning

Error Decomposition

Boosting

Decision Making

Gradient descent

Similarity Learning

Neural Networks Learning

Mathematical Optimization

Multiclass Learning

Neural network

COS 324 - Introduction to Machine Learning Princeton University Fall 2017 A thorough introduction to machine learning principles such as online learning, decision making, gradient-based learning, and empirical risk minimization. It also explores regression, classification, dimensionality reduction, ensemble methods, neural networks, and deep learning. The course material is self-contained and based on freely available resources. The course provides an introduction to machine learning.

**Topic covered**:

- Online learning and decision making
- Learning from examples and generalization
- Empirical risk minimization and regularization
- Introduction to convex analysis
- Gradient-based learning
- Implementation and analysis of learning algorithms for regression, binary classification, multiclass categorization, and ranking problems
- Dimensionality reduction methods
- Ensemble methods and boosting
- Neural networks and deep learning
- Markov decision precesses    **NOTICE:**  All material of the course is self-contained and based on freely available books and surveys.   
Main references:

- [Understanding Machine Learning: From Theory to Algorithms](http://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/), by Shai Shalev-Shwartz and Shai Ben-David
- [Online convex optimization](http://ocobook.cs.princeton.edu/), by Elad Hazan
- [Machine Learning](http://www.cs.cmu.edu/afs/cs.cmu.edu/user/mitchell/ftp/mlbook.html), by Tom Mitchell
- An Introduction to Computational Learning Theory, by Michael Kearns &amp; Umesh Vazirani
- Machine Learning: A Probabilistic Perspective, by Kevin Murphy,

Further advanced references: - [Convex Optimization](http://stanford.edu/~boyd/cvxbook/), by Stephen Boyd and Lieven Vandenberghe
- [Convex optimization: algorithms and complexity](https://arxiv.org/abs/1405.4980), by Sebastien Bubeck
- Artificial Intelligence: A Modern Approach, by Stuart Russell and Peter Norvig

Python Tutorials - [An interactive python tutorial](https://learnpython.org/) from LearnPython.com
- [Tutorial for Python 2.7](https://docs.python.org/2.7/tutorial/) from python.org
- [Tutorial for Python 3](https://docs.python.org/3/tutorial/) from python.org

COS 324 - Introduction to Machine Learning

Machine Learning

Online Learning

2 courses cover this concept

CS 294 - The Mathematics of Information and Data

COS 324 - Introduction to Machine Learning