Language Modeling

Text classification

Vector Embeddings

PyTorch

Finite-state machine

Sequence Labelling

Machine Translation

Sequence-to-sequence (Seq2Seq)

Morphology

Syntax

Conditional Random Fields (CRF)

Semantics

Linguistic Structure Prediction

Neural Machine Translation

Multinomial Logistic Regression

Weighted Finite-State Transducers

CSE 447 and 517 Natural Language Processing University of Washington Winter 2022 This course provides a comprehensive overview of Natural Language Processing (NLP), including core components like text classification, machine translation, and syntax analysis. It offers two project types: implementation problem-solving for CSE 447, and reproducing experiments from recent NLP papers for CSE 517.  Natural language processing (NLP) seeks to endow computers with the ability to intelligently process human language. NLP components are used in conversational agents and other systems that engage in dialogue with humans, automatic translation between human languages, automatic answering of questions using large text collections, the extraction of structured information from text, tools that help human authors, and many, many more. This course will teach you the fundamental ideas used in key NLP components. It is organized into several “greatest hits” topics, each with a more-or-less self-contained lecture and associated readings, problems, and implementation exercises.  The courses are similar in breadth and use the same lecture content. The projects are quite different; 447’s project is a predefined implementation problem that gives teams freedom in developing a solution. It is designed to encourage iterative improvement and an understanding of inherent tradeoffs in building an NLP system. 517’s project is more research-oriented; it asks teams to reproduce experiments in recently published NLP papers. Teams have great flexibility in the choice of a paper to reproduce.

Additionally, there will be differences in the assignments.

CSE 447 and 517 Natural Language Processing

Natural Language Processing

Regular Expressions

Minimum Edit Distance

Naive Bayes

Sentiment Analysis

Logistic Regression

Information Retrieval

Vector Semantics and Embeddings

Chatbots

Web graphs, Links, and PageRank

Social Networks

Neural network

CS 124: From Languages to Information Stanford University Winter 2023 This course is centered on extracting information from unstructured data in language and social networks using machine learning tools. It covers techniques like sentiment analysis, chatbot development, and social network analysis. The online world has a vast array of unstructured information in the form of language and social networks. Learn how to make sense of it using neural networks and other machine learning tools, and how to interact with humans via language, from answering questions to giving advice! Extracting meaning, information, and structure from human language text, speech, web pages, social networks. Introducing methods (string algorithms, edit distance, language modeling, machine learning, logistic regression, neural networks, neural embeddings, inverted indices, collaborative filtering, PageRank), applications (chatbots, sentiment analysis, information retrieval, text classification, social networks, recommender systems), and ethical issues. CS106B. CS 107 can be helpful, but is fine if you haven't had it, we'll cover the required UNIX material. Math 51 can also be helpful, but isn't required, since we will introduce the basic vectors knowledge we need in the class. #### Textbook

There is no required textbook, but I'll expect you to know the textbook/reading material listed above, and will test it on the midterms.
- Online new chapters from [Jurafsky and Martin. third edition in progress. *Speech and Language Processing*](https://web.stanford.edu/~jurafsky/slp3/).
- Chapters from Manning, Raghavan, and Schutze. 2008. Introduction to Information Retrieval. Cambridge University Press. You can buy the book, get it from the library, or it's also available online [*HERE*](https://site.ebrary.com/lib/stanford/docDetail.action?docID=10240274).

**Is 106B the only prereq? Do I need 109 or 221 before I take CS124?** : 106B is the only prereq. Taking the course as a sophomore is recommended, but we also get lots of juniors and a reasonable number of frosh; the course is designed to be taken early in your Stanford career. It will help if you have at least done some programming beyond 106B, and is also useful to have had 107 or Math 51, but not required; we'll try to give you pointers to places to make up missing background.

**Can I take this course as a non-CS grad student?**: Yes, although this course is not appropriate for CS grad students (there are graduate versions of all the material in this course), it's very commonly taken by PhD students in the social sciences or humanities who plan to use text processing methods in their research.

CS 124: From Languages to Information

K-Nearest Neighbors

Linear Classifiers

Stochastic gradient descent (SGD)

Momentum

AdaGrad

Adam

Learning Rate Schedules

Support Vector Machine (SVM)

Softmax Loss

Multi-layer Perceptron

Backpropagation

Higher-level representations

Image Features

Pooling

Batch Normalization

Transfer learning

AlexNet, VGG, GoogLeNet, ResNet

Activation Functions

Data Processing

Weight initialization

Hyperparameter Tuning

Data Augmentation

Feature visualization and inversion

Adversarial Examples

DeepDream and Style Transfer

Single-stage detectors

Two-stage detectors

Semantic/Instance/Panoptic segmentation

Long Short-Term Memory (LSTM)

Gated recurrent unit (GRU)

Image Captioning

Object Detection

Self-Attention

Transformer (machine learning model)

Video classification

3D CNNs

Two-stream networks

Multimodal Video Understanding

Supervised learning

Unsupervised learning

Pixel RNN, Pixel CNN

Variational autoencoder (VAE)

Generative adversarial network (GAN)

Pretext Tasks

Contrastive Learning

Multisensory Supervision

Optical Flow

Stereo Vision

3D Shape Representations

Shape Reconstruction

Neural Implicit Representations

Convolution

CS231n: Deep Learning for Computer Vision Stanford University Spring 2022 This is a deep-dive into the details of deep learning architectures for visual recognition tasks. The course provides students with the ability to implement, train their own neural networks and understand state-of-the-art computer vision research. It requires Python proficiency and familiarity with calculus, linear algebra, probability, and statistics. Computer Vision has become ubiquitous in our society, with applications in search, image understanding, apps, mapping, medicine, drones, and self-driving cars. Core to many of these applications are visual recognition tasks such as image classification, localization and detection. Recent developments in neural network (aka “deep learning”) approaches have greatly advanced the performance of these state-of-the-art visual recognition systems. This course is a deep dive into the details of deep learning architectures with a focus on learning end-to-end models for these tasks, particularly image classification. During the 10-week course, students will learn to implement and train their own neural networks and gain a detailed understanding of cutting-edge research in computer vision. Additionally, the final assignment will give them the opportunity to train and apply multi-million parameter networks on real-world vision problems of their choice. Through multiple hands-on assignments and the final course project, students will acquire the toolset for setting up deep learning tasks and practical engineering tricks for training and fine-tuning deep neural networks.  - Proficiency in Python  
    All class assignments will be in Python (and use numpy) (we provide a tutorial [here](http://cs231n.github.io/python-numpy-tutorial/) for those who aren't as familiar with Python). If you have a lot of programming experience but in a different language (e.g. C/C++/Matlab/Javascript) you will probably be fine.
- College Calculus, Linear Algebra (e.g. MATH 19 or 41, MATH 51)  
     You should be comfortable taking derivatives and understanding matrix vector operations and notation.
- Basic Probability and Statistics (e.g. CS 109 or other stats course)  
    You should know basics of probabilities, gaussian distributions, mean, standard deviation, etc. 

CS231n: Deep Learning for Computer Vision

Deep Learning

Computer Vision

Language model

Language Modeling

3 courses cover this concept

CSE 447 and 517 Natural Language Processing

CS 124: From Languages to Information

CS231n: Deep Learning for Computer Vision