Machine Translation

Information extraction

Words

Language models and smoothing

Lexicons

Morphology

Question Answering (QA)

Noisy channel models

Edit Distance

Part of speech tags

Sequence Tagging

Classification

Deep learning

Syntactic representations of natural language

Chomsky hierarchy and natural language

Treebanks

Lexical semantics

Vector semantics

Contextualized representations (BERT)

Verb / sentence semantics

Compositional semantics

Semantic Parsing

Discourse

Entity linking

Pragmatics

Speech

Word Embeddings

Natural Language Generation

11-411/611 Natural Language Processing Carnegie Mellon University Spring 2021 Focused on computational systems for human languages, this course introduces various NLP applications, such as translation and summarization. It encompasses a broad scope, from machine learning to linguistics, with a software engineering perspective. This course is about a variety of ways to represent human languages (like English and Chinese) as computational systems, and how to exploit those representations to write programs that do neat stuff with text and speech data, like

- translation,
- summarization,
- extracting information,
- question answering,
- natural interfaces to databases, and
- conversational agents.

This field is called Natural Language Processing or Computational Linguistics, and it is extremely multidisciplinary. This course will therefore include some ideas central to Machine Learning and to Linguistics.

We'll cover computational treatments of words, sounds, sentences, meanings, and conversations. We'll see how probabilities and real-world text data can help. We'll see how different levels interact in state-of-the-art approaches to applications like translation and information extraction.

From a software engineering perspective, there will be an emphasis on rapid prototyping, a useful skill in many other areas of Computer Science. ### Should I take this course?

Yes, if:

- you're a CS student interested in languages, language technology, or information processing
- you're a CS student who needs an "applications" credit
- you're a language technology minor (this course is an elective option)
- you're a linguistics student who can write computer programs (this course is an elective option)
- you always suspected natural language was kind of like Lisp (or Java or ...)
- you want computers to take over the world
- you don't want computers to take over the world, but if they do, you want to negotiate your release
- you like AI, machine learning, and/or theoretical computer science, and want to apply them to a hard real-world problem CS courses on data structures and algorithms, and strong programming skills. [Speech and Language Processing](https://home.cs.colorado.edu/~martin/slp2.html) (2nd Edition, 2007, Prentice-Hall), by Daniel Jurafsky and James Martin

### FAQ
Should I take this course?

Yes, if:

- you're a CS student interested in languages, language technology, or information processing
- you're a CS student who needs an "applications" credit
- you're a language technology minor (this course is an elective option)
- you're a linguistics student who can write computer programs (this course is an elective option)
- you always suspected natural language was kind of like Lisp (or Java or ...)
- you want computers to take over the world
- you don't want computers to take over the world, but if they do, you want to negotiate your release
- you like AI, machine learning, and/or theoretical computer science, and want to apply them to a hard real-world problem

11-411/611 Natural Language Processing

Natural Language Processing

Text classification

Language Modeling

Vector Embeddings

PyTorch

Finite-state machine

Sequence Labelling

Sequence-to-sequence (Seq2Seq)

Syntax

Conditional Random Fields (CRF)

Semantics

Linguistic Structure Prediction

Neural Machine Translation

Multinomial Logistic Regression

Weighted Finite-State Transducers

CSE 447 and 517 Natural Language Processing University of Washington Winter 2022 This course provides a comprehensive overview of Natural Language Processing (NLP), including core components like text classification, machine translation, and syntax analysis. It offers two project types: implementation problem-solving for CSE 447, and reproducing experiments from recent NLP papers for CSE 517.  Natural language processing (NLP) seeks to endow computers with the ability to intelligently process human language. NLP components are used in conversational agents and other systems that engage in dialogue with humans, automatic translation between human languages, automatic answering of questions using large text collections, the extraction of structured information from text, tools that help human authors, and many, many more. This course will teach you the fundamental ideas used in key NLP components. It is organized into several “greatest hits” topics, each with a more-or-less self-contained lecture and associated readings, problems, and implementation exercises.  The courses are similar in breadth and use the same lecture content. The projects are quite different; 447’s project is a predefined implementation problem that gives teams freedom in developing a solution. It is designed to encourage iterative improvement and an understanding of inherent tradeoffs in building an NLP system. 517’s project is more research-oriented; it asks teams to reproduce experiments in recently published NLP papers. Teams have great flexibility in the choice of a paper to reproduce.

Additionally, there will be differences in the assignments.

CSE 447 and 517 Natural Language Processing

N-gram Language Models

Sequence Models

Constituency Parsing

Dependency Parsing

Recurrent neural network (RNN)

Transformer (machine learning model)

Contextualized Representations and Pre-training

Large Language Model (LLM)

Long Short-Term Memory (LSTM)

COS 484: Natural Language Processing Princeton University Spring 2023 This course introduces the basics of NLP, including recent deep learning approaches. It covers a wide range of topics, such as language modeling, text classification, machine translation, and question answering. ### What is this course about?

Recent advances have ushered in exciting developments in natural language processing (NLP), resulting in systems that can translate text, answer questions and even hold spoken conversations with us. This course will introduce students to the basics of NLP, covering standard frameworks for dealing with natural language as well as algorithms and techniques to solve various NLP problems, including recent deep learning approaches. Topics covered include language modeling, representation learning, text classification, sequence tagging, syntactic parsing, machine translation, question answering and others.  - Required: [COS 324](https://www.cs.princeton.edu/courses/archive/fall21/cos324/), knowledge of probability, linear algebra, multivariate calculus.
- Proficiency in Python: programming assignments and projects will require use of Python, Numpy and PyTorch. ### Reading:

There is no required textbook for this class, and you should be able to learn everything from the lectures and assignments. However, if you would like to pursue more advanced topics or get another perspective on the same material, here are some books (all of them can be read free online):

- Dan Jurafsky and James H. Martin. [Speech and Language Processing (3rd ed. draft).](https://web.stanford.edu/~jurafsky/slp3/)
- Jacob Eisenstein. [Natural Language Processing ](https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstein-nlp-notes.pdf)
- Christopher Manning and Hinrich Schütze. [Foundations of Statistical Natural Language Processing](https://nlp.stanford.edu/fsnlp/).

COS 484: Natural Language Processing

Supervised learning

Linear regression

Training/Validation/Testing

Perceptron

MNIST

Loss function

Optimization

Backpropagation

Automatic differentiation

Matrix representation of Neural Networks

Graphics Processor (GPU)

TensorFlow

Multilayer perceptron

Activation Functions

Convolutional neural network (CNN)

Overfitting

Regularization (mathematics)

Language Models

Feedforward neural network

Gated recurrent unit (GRU)

Attention (machine learning)

Scaling Deep Learning Systems

Interpretation of Neural Networks

Unsupervised learning

Autoencoders

Variational autoencoder (VAE)

Generative adversarial network (GAN)

Deepfake

Reinforcement learning (RL)

Value iteration

Deep Q-learning

Policy Gradient Methods

Actor-Critic Methods

Graph neural network (GNN)

CSCI 1470/2470 Deep Learning Brown University Spring 2022 Brown University's Deep Learning course acquaints students with the transformative capabilities of deep neural networks in computer vision, NLP, and reinforcement learning. Using the TensorFlow framework, topics like CNNs, RNNs, deepfakes, and reinforcement learning are addressed, with an emphasis on ethical applications and potential societal impacts. Over the past few years, Deep Learning has become a popular area, with deep neural network methods obtaining state-of-the-art results on applications in computer vision (Self-Driving Cars), natural language processing (Google Translate), and reinforcement learning (AlphaGo). These technologies are having transformative effects on our society, including some undesirable ones (e.g. deep fakes).

This course is there to give students a practical understanding of how Deep Learning works, how to implement neural networks, and how to apply them ethically. We introduce students to the core concepts of deep neural networks and survey the techniques used to model complex processes within the contexts of computer vision and natural language processing.

Throughout the course, we emphasize and require students to think critically about potential ethical pitfalls that can result from mis-application of these powerful models. The course is taught using the Tensorflow deep learning framework.
 By the end of this course, you will be able to:

- Learn about the fundamental algorithms that underly all modern deep learning models.
- Implement different types of deep learning models in Tensorflow.
- Think critically about using a deep learning model for a task and its potential societal impact.
- Collaborate with classmates on a team project to apply deep learning models to task of your
choice.
- Communicate your findings (both positive and negative results are encouraged) through pre-
sentations. - A basic programming course: (CSCI 0150, 0170 or 0190)
- A linear algebra course: (CSCI 0530, MATH 0520 or 0540)
- A stats / probability course: (CSCI 0220, 1450, 0450, MATH 1610, APMA 1650 or 1655)

Exceptions may be possible for those missing one of these prerequisites if (a) the student has taken
another course which covers similar material, or if (b) the student will be concurrently taking the
prerequisite. If either of these situations applies to you, use the “Request Override” feature in
Courses@Brown to request an override code (and explain why you believe your situation merits
one). ### Textbook

None required. Students are encouraged to refer to the following textbook, which is available online:

- [Deep Learning](https://www.deeplearningbook.org/), by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.

CSCI 1470/2470 Deep Learning

Deep Learning