Empirical risk minimization

Property-preserving lossy compression

Nearest Neighbor

k-d tree

Regularization (mathematics)

L1 regularization

Matrix compression

Laplacian of a graph

Reservoir sampling

Markov chain Monte Carlo (MCMC)

Majority elements

Distance-preserving compression

PAC Guarantees

Linear-Algebraic Techniques

De-noising

Eigenvectors/eigenvalues

Markov's inequality

The Fourier Perspective

Linear programming

Dimensionality Reduction

Approximate heavy hitters

Locality-Sensitive Hashing (LSH)

MinHash

Principal Component Analysis (PCA)

Matrix completion

Spectral embeddings

Chebyshev‘s inequality

Convolution

Convex optimization

Bloom filters

Jaccard Similarity

Polynomial embedding

Maximizing variance

Uniqueness of low-rank tensor factorizations

Graph coloring

Importance Sampling

Sparse Vector/Matrix Recovery

Privacy Preserving Computation

Count-min sketch

Euclidean Distance

Johnson-Lindenstrauss Transform

Random projection

Power iteration algorithm

Jenrich's algorithm

Spectral clustering

Markov Chains

Compressed Sensing

Differential privacy

Similarity Search

Lp distance

Generalization

L2 regularization

Singular Value Decomposition (SVD)

Spectral Graph Theory

Interpretations of the second eigenvalue

Stationary distributions

Mathematical Programming

CS 168: The Modern Algorithmic Toolbox Stanford University Spring 2022 CS 168 provides a comprehensive introduction to modern algorithm concepts, covering hashing, dimension reduction, programming, gradient descent, and regression. It emphasizes both theoretical understanding and practical application, with each topic complemented by a mini-project. It's suitable for those who have taken CS107 and CS161.  This course will provide a rigorous and hands-on introduction to the central ideas and algorithms that constitute the core of the modern algorithms toolkit. Emphasis will be on understanding the high-level theoretical intuitions and principles underlying the algorithms we discuss, as well as developing a concrete understanding of when and how to implement and apply the algorithms. The course will be structured as a sequence of one-week investigations; each week will introduce one algorithmic idea, and discuss the motivation, theoretical underpinning, and practical applications of that algorithmic idea. Each topic will be accompanied by a mini-project in which students will be guided through a practical application of the ideas of the week. Topics include modern techniques in hashing, dimension reduction, linear and convex programming, gradient descent and regression, sampling and estimation, compressive sensing, and linear-algebraic techniques (principal components analysis, singular value decomposition, spectral techniques).  CS107 and CS161, or permission from the instructor. 

CS 168: The Modern Algorithmic Toolbox

Theoretical Computer Science

Connectionist Machines

McCullough and Pitt model

Hebb’s learning rule

Rosenblatt’s perceptron

Universal Approximator

Neural network

Perceptron learning rule

Optimization

Gradient descent

Back-propagation

Momentum

Nestorov

Convergence

Learning Rates

Optimization Algorithms

RMSProp

Stochastic gradient descent (SGD)

Acceleration

Overfitting

Convolutional neural network (CNN)

Translation Invariance

Cascade Correlation Filters

Recurrent neural network (RNN)

Bidirectional RNNs

Sequence Prediction

Long Short-Term Memory (LSTM)

Connectionist Temporal Classification (CTC)

Representations

Autoencoders

Hopfield Networks

Boltzmann Machines

Normalizing Flows

Variational autoencoder (VAE)

Generative adversarial network (GAN)

Reinforcement learning (RL)

Multi-layer Perceptron

Sequence-to-sequence (Seq2Seq)

AdaGrad

11-785 Introduction to Deep Learning Carnegie Mellon University Spring 2020 This course provides a comprehensive introduction to deep learning, starting from foundational concepts and moving towards complex topics such as sequence-to-sequence models. Students gain hands-on experience with PyTorch and can fine-tune models through practical assignments. A basic understanding of calculus, linear algebra, and Python programming is required. “Deep Learning” systems, typified by deep neural networks, are increasingly taking over all AI tasks, ranging from language understanding, and speech and image recognition, to machine translation, planning, and even game playing and autonomous driving. As a result, expertise in deep learning is fast changing from an esoteric desirable to a mandatory prerequisite in many advanced academic settings, and a large advantage in the industrial job market.

In this course we will learn about the basics of deep neural networks, and their applications to various AI tasks. By the end of the course, it is expected that students will have significant familiarity with the subject, and be able to apply Deep Learning to a variety of tasks. They will also be positioned to understand much of the current literature on the topic and extend their knowledge through further study.

If you are only interested in the lectures, you can watch them on the YouTube channel listed below.

### Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.  1. We will be using one of several toolkits (the primary toolkit for recitations/instruction is PyTorch). The toolkits are largely programmed in Python. You will need to be able to program in at least one of these languages. Alternately, you will be responsible for finding and learning a toolkit that requires programming in a language you are comfortable with,
1. You will need familiarity with basic calculus (differentiation, chain rule), linear algebra and basic probability. ### Course description from student point of view

The course is well rounded in terms of concepts. It helps us understand the fundamentals of Deep Learning. The course starts off gradually with MLPs and it progresses into the more complicated concepts such as attention and sequence-to-sequence models. We get a complete hands on with PyTorch which is very important to implement Deep Learning models. As a student, you will learn the tools required for building Deep Learning models. The homeworks usually have 2 components which is Autolab and Kaggle. The Kaggle components allow us to explore multiple architectures and understand how to fine-tune and continuously improve models. The task for all the homeworks were similar and it was interesting to learn how the same task can be solved using multiple Deep Learning approaches. Overall, at the end of this course you will be confident enough to build and tune Deep Learning models.

11-785 Introduction to Deep Learning

Deep Learning

Empirical risk minimization

2 courses cover this concept

CS 168: The Modern Algorithmic Toolbox

11-785 Introduction to Deep Learning