Multithreading

System Calls

Multiprocessing

execvp

Interprocess Communication

Virtual memory

Thread (computing)

Mutexes (Mutual exclusion)

Semaphores

Thread Pools

Parallel Compilation

Clients

Networking

API (Application Programming Interface)

MapReduce

Filesystems

File Descriptors

waitpid

Pipeline (Unix)

Signals

Race condition

Condition variables

Multithreading Patterns

Read-Write Locks

Networks

Servers

HTTP

Networking Functions

CS 110: Principles of Computer Systems Stanford University Winter 2022 CS 110 delves into advanced computer systems and program construction, focusing on designing large systems, software that spans multiple machines, and parallel computing. This course builds upon CS107 and requires good knowledge of C, C++, Unix, GDB, Valgrind, and Make. It covers Linux filesystems, multiprocessing, threading, networking, and more. CS107 took you behind the scenes:

- how things work inside C++/Python/Java, and how your programs map onto the components of computer systems
- understanding of program behavior and execution
 
CS110 uses this as a foundation to build complex programs that maximally take advantage of the hardware and operating system software available to us:

- How can we understand the designs and tradeoffs of large systems?
- How can we write software that spans multiple machines?
- How can we write software that runs tasks in parallel on a single machine? ### Course Topics Overview

1. **Overview of Linux Filesystems** - How can we design filesystems to store and manipulate files on disk?
1. **Multiprocessing and Exceptional Control Flow** - How can our program create and interact with other programs?
1. **Threading and Concurrency** - How can a single instance of our program perform multiple coordinated tasks at the same time?
1. **Networking and Distributed Computing** - How can we write programs that communicate over a network with other programs, and tackle large tasks using many machines?
1. **Additional Topics**: MapReduce, Caching, and Non-Blocking I/O **CS107 or equivalent -**

- Each of you should know C and C++ reasonably well so that you can...
    - write moderately complex programs (e.g. pointers, **malloc/realloc/free**, C strings, C++ classes, methods, references, templates, **new/delete**)
    - read and understand portions of large code bases
    - trace memory diagrams and always win!
- Each of you should be ﬂuent with **Unix, GDB, Valgrind**, and **Make** to the extent they're covered in CS107 or its equivalent.

The first assignment is meant to give you a sense of the scope of CS110 programs and refresh your memory on relevant prerequisites.  If you feel ok about it, you're all set! Second half of CS107 Textbook: *Computer Systems: A Programmer's Perspective by Bryant & O'Hallaron, 3rd Edition*

- Can purchase full copy, or Stanford Bookstore custom edition with just CS110 chapters
 
*Principles of Computer System Design: An Introduction* by Jerome H. Saltzer and M. Frans Kaashoek

- Free Stanford online access [here](https://www.sciencedirect.com/book/9780123749574/principles-of-computer-system-design)​ (also linked on course homepage)
- Fewer readings from this book vs. first one
- You're welcome to buy a physical copy if you'd like

CS 110: Principles of Computer Systems

Computer Systems

ARM processor and architecture

C Functions

C Mastery

Interrupts

From Assembly to C

Libraries

PS/2 Protocol

Computer Arithmetic

C Pointers

Linking

Graphics

C Arrays

Framebuffer

Memory Systems

ARM assembly and machine code

Modules

Keyboards

Sensors

Memory management

CS 107e Computer Systems from the Ground Up Stanford University Winter 2023 CS 107e focuses on bare metal programming on the Raspberry Pi, serving as an introduction to embedded systems. It covers the entire process from the microprocessor to the C programming language. The course aims to build a solid understanding of how modern computers execute programs and how program development tools work.  CS107 is the third course in Stanford's introductory programming sequence. The CS106A/B courses provide students with a solid foundation in programming methodology and abstractions, and CS107 follows on to develop the skills needed to build computer systems.

**CS107e: Computer Systems from the Ground Up** is a variant of CS107 that teaches the fundamental concepts of computer systems through bare metal programming on the Raspberry Pi. Bare metal programming means you will not run an operating system on the Raspberry Pi and will make minimal use of libraries. This course also serves as an introduction to embedded systems. The course starts with the microprocessor and moves up to the C programming language, without skipping anything in between. The goal is to build a solid understanding of all aspects of how modern computers execute programs and how program development tools work. The major learning goals for the course are:

- To understand how computers represent information, execute programs, and control peripherals.
- To master command-line programming tools and the C programming language.

Topics covered include:

- the ARM architecture
- assembly language and machine-level code
- the C programming language
- compilation, linking and loading
- debugging
- memory organization and management
- controlling peripherals: GPIO, graphics, sound, and keyboards  

CS 107e Computer Systems from the Ground Up

Microcode

Pipelining

Superscalar

Very long instruction word (VLIW)

Branch Prediction

Memory protection

Graphics Processor (GPU)

Parallel Programming

Superpipelining

Vector Processors

Small Multiprocessors

Multiprocessor Interconnect

Large Multiprocessors

Directory Protocols

Set Architectures

Verilog

Exception handling

Cache (computing)

ELE/COS 475 Computer Architecture Princeton University Fall 2019 This course offers an in-depth understanding of modern computer processor and system architecture. It covers topics like instruction-set architecture, processor organization, cache, memory, multiprocessors, and more. Designed for senior-level undergraduates and first-year graduate students. An in-depth study of the fundamentals of modern computer processor and system architecture. Students will develop a strong theoretical and practical understanding of modern, cutting-edge computer architectures and implementations. Studied topics include: Instruction-set architecture and high-performance processor organization including pipelining, out-of-order execution, as well as data and instruction parallelism. Cache, memory, and storage architectures. Multiprocessors and multicore processors. Coherent caches. Interconnection and network infrastructures. If the course indicates that enrollment capacity has been reached, students are encouraged to come to the class and speak with the professor about enrollment. Approximately 20 pages of reading per week. Homework - 5 problem sets during the semester.  This course is targeted at senior-level undergraduates and first-year graduate students. Students should have a good working understanding of digital logic, basic processor design and organization, pipelining, and simple cache design. ELE 375/COS 375 and ELE 206/COS 306 or equivalent knowledge are necessary to succeed in ELE 475. Use of the Verilog hardware description language will be used in this course and prior knowledge of Verilog will be very helpful, but not required. **Required text:**

1.  Computer Architecture: A Quantitative Approach 5th or 6th edition
    -  John L. Hennessy and David A. Patterson
    -  Princeton University Library Owns

**Recommended Text:**

-  Modern Processor Design: Fundamentals of Superscalar Processors (1st Edition), 2004. First and second printing are the same.
    -  John P. Shen and Mikko H. Lipasti
    -  Approximately $68 on [Amazon](http://www.amazon.com/Modern-Processor-Design-Fundamentals-Superscalar/dp/1478607831/ "http://www.amazon.com/Modern-Processor-Design-Fundamentals-Superscalar/dp/1478607831/")
    -  ISBN: 978-1478607830
    -  Princeton University Library Owns

**Other Text:**

-  Verilog HDL: A Guide to Digital Design and Synthesis, (2nd Edition), 2003.
    -  Samir Palnitkar
    -  ISBN: 0-13-044911-3
    -  This book is a good Verilog reference as we will be using Verilog in the labs for this course. If you already know Verilog for synthesis than this may not be needed. The 1st and 2nd editions of this book work. Likewise there are many other Verilog references on the web and other books are available.
    -  The 1st edition can be purchased used from Amazon for less than $10, the 2nd edition is more expensive.
    -  Princeton University Library has free online access to this book: [Online Safari Version On Princeton Network](http://proquest.safaribooksonline.com/?uiCode=princeun&xmlId=0130449113 "http://proquest.safaribooksonline.com/?uiCode=princeun&xmlId=0130449113")

ELE/COS 475 Computer Architecture

Lock (computer science)

Parallel computing

Parallel efficiency

Multi-core processor

SIMD (Single Instruction, Multiple Data)

Latency

Bandwidth (computing)

ISPC (Intel SPMD Program Compiler)

Work Stealing

Message Passing

Asynchronous Communication

Arithmetic Intensity

Contention

GPU Architecture

CUDA (Compute Unified Device Architecture)

Data Parallelism

Prefix Sum

Apache Spark

Resilient Distributed Dataset (RDD)

Cache coherence

MSI Protocol

MESI Protocol

False Sharing

Memory Consistency

Non-blocking algorithm

Transactional memory

Software Transactional Memory (STM)

Hardware Transactional Memory (HTM)

Energy-Efficient Computing

Heterogeneous Computing

Fixed-function

Field-Programmable Gate Array (FPGA)

System on a Chip (SoC)

Domain-Specific Language (DSL)

Halide (programming language)

Graph Processing

DRAM (Dynamic Random-Access Memory)

Reconfigurable Computing

Deep Neural Network

Matrix Multiplication

Layer Fusion

TensorCore

Tensor Processing Unit (TPU)

CS 149 PARALLEL COMPUTING Stanford University  Fall 2022 Focused on principles and trade-offs in designing modern parallel computing systems, this course also teaches parallel programming techniques. It is intended for students looking to understand both parallel hardware and software design. Prerequisite knowledge in computer systems is required. From smart phones, to multi-core CPUs and GPUs, to the world's largest supercomputers and web sites, parallel processing is ubiquitous in modern computing. The goal of this course is to provide a deep understanding of the fundamental principles and engineering trade-offs involved in designing modern parallel computing systems as well as to teach parallel programming techniques necessary to effectively utilize these machines. Because writing good parallel programs requires an understanding of key machine performance characteristics, this course will cover both parallel hardware and software design.  [CS110: Principles of Computer Systems](http://web.stanford.edu/class/cs110/) is strong prerequisite for CS149. If you have not taken CS110 and feel like you have the background to take CS149, please contact your friendly instructors and let's talk it over. There is no required textbook for CS149. However, you may find the following textbooks helpful. However, in general these days there's plenty of great parallel programming resources available for free on the web.

John L. Hennessy and David A. Patterson

Computer Architecture, Sixth Edition: A Quantitative Approach. Morgan Kaufmann, 2017

[[ On Amazon ]](https://www.amazon.com/Computer-Architecture-Quantitative-Approach-Kaufmann/dp/0128119055)


Jason Sanders

CUDA by Example: An Introduction to General-Purpose GPU Programming. 1st Edition. Addison-Wesley. 2010

[[ On Amazon ]](https://www.amazon.com/CUDA-Example-Introduction-General-Purpose-Programming/dp/0131387685)


David Kirk and Wen-mei Hwu

Programming Massively Parallel Processors, Second Edition: A Hands-on Approach. 2nd Edition. Morgan Kauffmann. 2012

[[ On Amazon ]](https://www.amazon.com/Programming-Massively-Parallel-Processors-Second/dp/0124159923)

CS 149 PARALLEL COMPUTING

Parallelism and Distributed Systems

Multithreading (computer architecture)

Multithreading

4 courses cover this concept

CS 110: Principles of Computer Systems

CS 107e Computer Systems from the Ground Up

ELE/COS 475 Computer Architecture

CS 149 PARALLEL COMPUTING