Deep learning

Information extraction

Words

Language models and smoothing

Lexicons

Morphology

Question Answering (QA)

Noisy channel models

Edit Distance

Part of speech tags

Sequence Tagging

Classification

Syntactic representations of natural language

Chomsky hierarchy and natural language

Treebanks

Lexical semantics

Vector semantics

Contextualized representations (BERT)

Verb / sentence semantics

Compositional semantics

Semantic Parsing

Discourse

Entity linking

Pragmatics

Speech

Word Embeddings

Natural Language Generation

Machine Translation

11-411/611 Natural Language Processing Carnegie Mellon University Spring 2021 Focused on computational systems for human languages, this course introduces various NLP applications, such as translation and summarization. It encompasses a broad scope, from machine learning to linguistics, with a software engineering perspective. This course is about a variety of ways to represent human languages (like English and Chinese) as computational systems, and how to exploit those representations to write programs that do neat stuff with text and speech data, like

- translation,
- summarization,
- extracting information,
- question answering,
- natural interfaces to databases, and
- conversational agents.

This field is called Natural Language Processing or Computational Linguistics, and it is extremely multidisciplinary. This course will therefore include some ideas central to Machine Learning and to Linguistics.

We'll cover computational treatments of words, sounds, sentences, meanings, and conversations. We'll see how probabilities and real-world text data can help. We'll see how different levels interact in state-of-the-art approaches to applications like translation and information extraction.

From a software engineering perspective, there will be an emphasis on rapid prototyping, a useful skill in many other areas of Computer Science. ### Should I take this course?

Yes, if:

- you're a CS student interested in languages, language technology, or information processing
- you're a CS student who needs an "applications" credit
- you're a language technology minor (this course is an elective option)
- you're a linguistics student who can write computer programs (this course is an elective option)
- you always suspected natural language was kind of like Lisp (or Java or ...)
- you want computers to take over the world
- you don't want computers to take over the world, but if they do, you want to negotiate your release
- you like AI, machine learning, and/or theoretical computer science, and want to apply them to a hard real-world problem CS courses on data structures and algorithms, and strong programming skills. [Speech and Language Processing](https://home.cs.colorado.edu/~martin/slp2.html) (2nd Edition, 2007, Prentice-Hall), by Daniel Jurafsky and James Martin

### FAQ
Should I take this course?

Yes, if:

- you're a CS student interested in languages, language technology, or information processing
- you're a CS student who needs an "applications" credit
- you're a language technology minor (this course is an elective option)
- you're a linguistics student who can write computer programs (this course is an elective option)
- you always suspected natural language was kind of like Lisp (or Java or ...)
- you want computers to take over the world
- you don't want computers to take over the world, but if they do, you want to negotiate your release
- you like AI, machine learning, and/or theoretical computer science, and want to apply them to a hard real-world problem

11-411/611 Natural Language Processing

Natural Language Processing

Memory hierarchy

Matrix Multiplication

Shared Memory Parallelism

Roofline and Performance Modeling

Sources of Parallelism and Locality

Communication-avoiding matrix multiplication

Data Parallel Algorithms

CUDA (Compute Unified Device Architecture)

Graphics Processor (GPU)

Distributed Memory Machines and Programming

Advanced MPI

Collective Communication Algorithms

UPC++: Partitioned Global Address Space Languages

Domain Specific Languages (Halide)

Distributed Data Structures (BCL)

Parallel Matrix Multiply

Dense Linear Algebra

Sparse-Matrix-Vector-Multiplication

Iterative Solvers

Structured Grids

Machine learning

Supervised learning

Unsupervised learning

Graph Partitioning

Fast Fourier transform

Graph Algorithms

Cloud Computing

Big Data Processing

High Performance Computing (HPC)

Dynamic Load Balancing

Hierarchical Methods for the N-Body Problem

Sorting algorithm

Search algorithm

Computational Biology

Quantum Computing

CS 267: Applications of Parallel Computers UC Berkeley Spring 2020 The course addresses programming parallel computers to solve complex scientific and engineering problems. It covers an array of parallelization strategies for numerical simulation, data analysis, and machine learning, and provides experience with popular parallel programming tools. CS267 was originally designed to teach students how to program parallel computers to efficiently solve challenging problems in science and engineering, where very fast computers are required either to perform complex simulations or to analyze enormous datasets. CS267 is intended to be useful for students from many departments and with different backgrounds, although we will assume reasonable programming skills in a conventional (non-parallel) language, as well as enough mathematical skills to understand the problems and algorithmic solutions presented. CS267 satisfies part of the course requirements for the Designated Emphasis ("graduate minor") in [Computational Science and Engineering](http://www.google.com/url?q=http%3A%2F%2Fcitris-uc.org%2Finitiatives%2Fdecse%2F&sa=D&sntz=1&usg=AOvVaw1fRFGvIaIYhkwE3kp4J1nf).

While this general outline remains, a large change in the computing world started in the mid 2000's: not only are the [fastest computers](http://www.google.com/url?q=http%3A%2F%2Fwww.top500.org%2F&sa=D&sntz=1&usg=AOvVaw1SiByfUjOpTR6ctURAV2Ef) parallel, but nearly all computers are becoming parallel, because the physics of semiconductor manufacturing will no longer let conventional sequential processors get faster year after year, as they have for so long (roughly doubling in speed every 18 months for many years). So all programs that need to run faster will have to become parallel programs. (It is considered very unlikely that compilers will be able to automatically find enough parallelism in most sequential programs to solve this problem.) For background on this trend toward parallelism, click [here](http://www.google.com/url?q=http%3A%2F%2Fwww.eecs.berkeley.edu%2FPubs%2FTechRpts%2F2006%2FEECS-2006-183.html&sa=D&sntz=1&usg=AOvVaw2XoPTheTzKVlyjp1G_YcM5).

Students in CS267 will get an overview of the parallel architecture space, gain experience using some of the most popular parallel programming tools, and be exposed to a number of open research questions. The lectures will also cover a broad set of parallelization strategies for applications covering numerical simulation and data analysis to machine learning.   

CS 267: Applications of Parallel Computers

Parallelism and Distributed Systems

Artificial intelligence

Health care

Medical imaging

Object detection

3D modeling

Video processing

NumPy

TensorFlow

Electronic health record

Multimodal Data

Multimodal Models

Genomics

Interpretability (machine learning)

Fairness (machine learning)

Ethics of AI and Healthcare

Distributed learning

Differential privacy

k-anonymity

Private aggregation of teacher ensembles (PATE)

Transformer (machine learning model)

Federated learning

Image Segmentation

Reinforcement learning (RL)

Self-supervised Learning

CS 271 / BIOMEDIN 220 Artificial Intelligence in Healthcare Stanford University Fall 2022-2023 Offered by Stanford University, this course focuses on AI applications in healthcare, exploring deep learning models for image, text, multimodal, and time-series data in the healthcare context. Topics also address AI integration challenges like interpretability and privacy. Healthcare is one of the most exciting application domains of artificial intelligence, with transformative potential in areas ranging from medical image analysis to electronic health records-based prediction and precision medicine. This course will involve a deep dive into recent advances in AI in healthcare, focusing in particular on deep learning approaches for healthcare problems. We will start from foundations of neural networks, and then study cutting-edge deep learning models in the context of a variety of healthcare data including image, text, multimodal and time-series data. In the latter part of the course, we will cover advanced topics on open challenges of integrating AI in a societal application such as healthcare, including interpretability, robustness, privacy and fairness. The course aims to provide students from diverse backgrounds with both conceptual understanding and practical grounding of cutting-edge research on AI in healthcare.  Proficiency in Python, or significant experience with a different programming language and ability to self-learn. Python will be used for homework assignments and the course project. Basic familiarity with college calculus (e.g. Math 19 or 41, comfortable taking derivatives), linear algebra (e.g. Math 51 or EE 103 / CME 103, comfortable with common matrix vector operations and notation), and probability and statistics (e.g. CME 106 or CS 109, comfortable with common probability distributions, mean, standard deviation, etc). Familiarity with machine learning, e.g. comfortable with the framework of machine learning and experience training a machine learning model. Familiarity with deep learning is highly recommended, e.g. prior experience training a deep learning model. 

CS 271 / BIOMEDIN 220 Artificial Intelligence in Healthcare

Artificial Intelligence

Interdisciplinary

Deep learning

3 courses cover this concept

11-411/611 Natural Language Processing

CS 267: Applications of Parallel Computers

CS 271 / BIOMEDIN 220 Artificial Intelligence in Healthcare