Naive Bayes

Regular Expressions

Minimum Edit Distance

Sentiment Analysis

Logistic Regression

Information Retrieval

Vector Semantics and Embeddings

Sequence Labelling

Chatbots

Web graphs, Links, and PageRank

Social Networks

Language Modeling

Text classification

Neural network

CS 124: From Languages to Information Stanford University Winter 2023 This course is centered on extracting information from unstructured data in language and social networks using machine learning tools. It covers techniques like sentiment analysis, chatbot development, and social network analysis. The online world has a vast array of unstructured information in the form of language and social networks. Learn how to make sense of it using neural networks and other machine learning tools, and how to interact with humans via language, from answering questions to giving advice! Extracting meaning, information, and structure from human language text, speech, web pages, social networks. Introducing methods (string algorithms, edit distance, language modeling, machine learning, logistic regression, neural networks, neural embeddings, inverted indices, collaborative filtering, PageRank), applications (chatbots, sentiment analysis, information retrieval, text classification, social networks, recommender systems), and ethical issues. CS106B. CS 107 can be helpful, but is fine if you haven't had it, we'll cover the required UNIX material. Math 51 can also be helpful, but isn't required, since we will introduce the basic vectors knowledge we need in the class. #### Textbook

There is no required textbook, but I'll expect you to know the textbook/reading material listed above, and will test it on the midterms.
- Online new chapters from [Jurafsky and Martin. third edition in progress. *Speech and Language Processing*](https://web.stanford.edu/~jurafsky/slp3/).
- Chapters from Manning, Raghavan, and Schutze. 2008. Introduction to Information Retrieval. Cambridge University Press. You can buy the book, get it from the library, or it's also available online [*HERE*](https://site.ebrary.com/lib/stanford/docDetail.action?docID=10240274).

**Is 106B the only prereq? Do I need 109 or 221 before I take CS124?** : 106B is the only prereq. Taking the course as a sophomore is recommended, but we also get lots of juniors and a reasonable number of frosh; the course is designed to be taken early in your Stanford career. It will help if you have at least done some programming beyond 106B, and is also useful to have had 107 or Math 51, but not required; we'll try to give you pointers to places to make up missing background.

**Can I take this course as a non-CS grad student?**: Yes, although this course is not appropriate for CS grad students (there are graduate versions of all the material in this course), it's very commonly taken by PhD students in the social sciences or humanities who plan to use text processing methods in their research.

CS 124: From Languages to Information

Natural Language Processing

Informed Search

Probability

Decision Networks

Perceptron

Search with Other Agents

Hidden Markov Model (HMM)

Optimization

Bayes' Nets: Independence

Constraint Satisfaction Problem (CSP)

Value of Information (VPI)

Bayes' Nets: Representation

Markov Decision Process (MDP)

Particle Filtering

Bayes' Nets: Inference

Uninformed Search

Reinforcement learning (RL)

Bayes' Nets: Sampling

CS 188 Introduction to Artificial Intelligence UC Berkeley Fall 2022 UC Berkeley's CS 188 course covers the basic ideas and techniques for designing intelligent computer systems, emphasizing statistical and decision-theoretic modeling. By the course's end, students will have built autonomous agents that can make efficient decisions in a variety of settings. This course will introduce the basic ideas and techniques underlying the design of intelligent computer systems. A specific emphasis will be on the statistical and decision-theoretic modeling paradigm.

By the end of this course, you will have built autonomous agents that efficiently make decisions in fully informed, partially observable and adversarial settings. Your agents will draw inferences in uncertain environments and optimize actions for arbitrary reward structures. Your machine learning algorithms will classify handwritten digits and photographs. The techniques you learn in this course apply to a wide variety of artificial intelligence problems and will serve as the foundation for further study in any application area you choose to pursue.

See the syllabus for slides, deadlines, and the lecture schedule. Readings refer to [fourth edition of AIMA](http://aima.cs.berkeley.edu/) unless otherwise specified.  - **CS 61A or 61B**: Prior computer programming experience is expected (see below)
- **CS 70 or Math 55**: Familiarity with basic concepts of propositional logic and probability are expected (see below)

*CS61A AND CS61B AND CS70 is the recommended background.*

The required math background in the second half of the course will be significantly greater than the first half. The self-diagnostic assignment Homework 0 will help check your preparation.

Course programming assignments will be in Python. We do not assume that students have previous experience with the language, but we do expect you to learn the basics very rapidly. Project 0 is designed to teach you the basics of Python and how the CS 188 submission autograder works. Project 1 is a good representation of the programming level that will be required for subsequent projects in this class. 

CS 188 Introduction to Artificial Intelligence

Artificial Intelligence

Decision Tree Learning

Margins

Maximizing Conditional Likelihood

Kernelizing Algorithms

Primal and Dual Forms

Finite Hypothesis Classes

k-fold cross-validation

Convolutional neural network (CNN)

Objective-Based Clustering

Principal Component Analysis (PCA)

Semi-Supervised Learning

Learning Linear Separators

Maximum Likelihood Estimation (MLE)

Bag of Words

Computer Vision

Generalization and Overfitting

Model Selection and Regularization

Support Vector Machine (SVM)

Minimizing Squared Error

Deep Networks

Adaboost

Learning Representations

Active Learning

Co-training

Q-learning

Overfitting

Conditional Independence

Gradient descent

Kernelizing Perceptron

Kernelizing SVM

VC Dimension Based Bounds

Linear regression

Backpropagation

Boosting

Hierarchical Clustering

Interactive Learning

Transductive SVM

Value iteration

Bayes' theorem

Maximum A Posteriori (MAP)

Kernel (operating system)

Geometric Margins

Sample Complexity

Structural Risk Minimization

Maximizing Data Likelihood

Unsupervised learning

Dimensionality Reduction

Sampling Bias

Convolution

10-401 Introduction to Machine Learning Carnegie Mellon University Spring 2018 A comprehensive exploration of machine learning theories and practical algorithms. Covers a broad spectrum of topics like decision tree learning, neural networks, statistical learning, and reinforcement learning. Encourages hands-on learning via programming assignments.  Machine Learning is concerned with computer programs that automatically improve their performance through experience (e.g., programs that learn to recognize human faces, recommend music and movies, and drive autonomous robots). This course covers the theory and practical algorithms for machine learning from a variety of perspectives. We cover topics such as decision tree learning, Support Vector Machines, neural networks, boosting, statistical learning methods, unsupervised learning, active leaerning, and reinforcement learning. Short programming assignments include hands-on experiments with various learning algorithms.   - Machine Learning, Tom Mitchell. (optional)
- Pattern Recognition and Machine Learning, Christopher Bishop. (optional)
- Machine Learning: A Probabilistic Perspective, Kevin P. Murphy, [available online](https://ebookcentral.proquest.com/lib/cm/detail.action?docID=3339490), (optional)

10-401 Introduction to Machine Learning

Machine Learning

Counting

Combinatorics

Conditional probability

Independence (probability theory)

Random variable

Expectation

Variance

Bernoulli distribution

Binomial distribution

Poisson distribution

Continuous Random Variables

Normal distribution

Joint probability distribution

Statistical inference

Beta distribution

Central Limit Theorem

P-Value

Algorithmic Analysis

Bootstrapping

CS 109 Probability for Computer Scientists Stanford University Spring 2023 This course offers a thorough understanding of probability theory and its applications in data analysis and machine learning. Prerequisites include CS103, CS106B, and Math 51 or equivalent courses. CS109: Probability for Computer Scientists starts by providing a fundamental grounding in combinatorics, and then quickly moves into the basics of probability theory. We will then cover many essential concepts in probability theory, including common probability distributions, properties of probabilities, and mathematical tools for analyzing probabilities. The last third of the class will focus on data analysis and machine learning as direct applications of probability we've learned in the weeks prior. Read more here to learn what CS109 is all about. This is going to be a great quarter and we are looking forward to the chance to teach you!! After you're finished with CS109, we hope you'll have achieved the following learning goals:

- Reason about situations using probabilities, expectation, and variance.
- Feel comfortable learning about new probability concepts beyond the scope of this class (e.g., through one's own research, studies, or interests).
- Write programs to simulate random experiments and to test experiment hypotheses.
- Understand and implement simple machine learning algorithms like Naive Bayes and Logistic Regression.

## Course Topics

Here are the broad strokes of the course (in approximate order). More information is available on our Schedule page. We cover a very broad set of topics so that you are equipped with the probability and statistics you will see in your future CS studies!

- Counting and probability fundamentals
- Single-dimensional random variables
- Probabilistic models
- Uncertainty theory
- Parameter estimation
- Introduction to machine learning The prerequisites for this course are CS103, CS106B, and Math 51 (or equivalent courses). Probability involves a fair bit of mathematics (set theory, calculus, and familiarity with linear algebra), and we'll be considering several applications of probability in CS that require familiarity with algorithms and data structures covered in CS106B. Here is a quick rundown of some of the mathematical tools from CS103 and Math 51 that we'll be using in this class: multivariate calculus (integration and differentiation), linear algebra (basic operations on vectors and matrices), an understanding of the basics of set theory (subsets, complements, unions, intersections, cardinality, etc.), and familiarity with basic proof techniques. We'll also do combinatorics in the class, but we'll be covering a fair bit of that material ourselves in the first week. Past students have managed to take CS106B concurrently with CS109 and have done just fine. CS103 is the pre-requisite that we rely on the least. See our FAQ for more information. ## Optional Textbook

Sheldon Ross, *A First Course in Probability* (10th Ed.), Pearson Prentice Hall, 2018.

This is an optional textbook, meaning that the text is not required material, but students may find Ross offers a different and useful perspective on the important concepts of the class. Suggested, optional reading assignments from the textbook (10th Ed.) are in the schedule on the course website. The 8th, 9th, and 10th editions of the textbook are all fine for this class.

*Borrowing the textbook online*: HathiTrust, a library archive of which Stanford is a member, has granted the university online access to the 8th edition (2010) for the duration of the Fall quarter. The "check out" system works similarly to print reserves: A user can check out the book an hour at a time as long as they are actively using it. Access guidelines are on the [HathiTrust How To Use It webpage](https://www.hathitrust.org/ETAS-How-To). Once you're logged in, the book is at [this link](https://catalog.hathitrust.org/Record/006904347).

All students should retain receipts for books and other course-related expenses, as these may be qualified educational expenses for tax purposes. If you are an undergraduate receiving financial aid, you may be eligible for additional financial aid for required books and course materials if these expenses exceed the aid amount in your award letter. For more information, review your award letter or visit the [Student Budget website](https://financialaid.stanford.edu/undergrad/budget/index.html).

CS 109 Probability for Computer Scientists

Mathematical Foundations

Naive Bayes classifier

Naive Bayes

4 courses cover this concept

CS 124: From Languages to Information

CS 188 Introduction to Artificial Intelligence

10-401 Introduction to Machine Learning

CS 109 Probability for Computer Scientists