Parallel Programming @ NCTU, Fall 2020
Announcement
- Presentation Notes (HackMD)
- Sign up for the presentation schedule at https://ppt.cc/fzWjrx by Dec. 3. More details are available here.
- Score sheet: Link
- Workstations are opened for testing. Checkout your email or new E3 platform for the detail.
- Please fill the grouping form (due: 9/30): Link
- The first lecture will start at 10:10 on September 17.
- Please Join the Facebook group of the class.
General Information
Class time: 4CDX (10:00-13:00 on Thursdays)
Class location: EDB27
Facebook group: Link
Refer to the SYLLABUS for more details.
People
| Member | Name | Office hours | Location | |
|---|---|---|---|---|
| Instructor | Prof. Yi-Ping You (游逸平) | ypyou [at] cs.nctu.edu.tw | By appointment | EC532 |
| TA | An-Chi Liu (劉安齊) | acliu [at] cs.nctu.edu.tw | By appointment | EC618 |
| TA | Yu-Sheng Hsieh (謝宇勝) | yshsieh [at] cs.nctu.edu.tw | By appointment | EC618 |
| TA | Yi-Yang Chao (趙益揚) | yychao [at] cs.nctu.edu.tw | By appointment | EC618 |
| TA | Yi-An Chen (陳奕安) | yachen [at] cs.nctu.edu.tw | By appointment | EC618 |
You may also mail to PP-f20 [at] sslab.cs.nctu.edu.tw. All TAs will receive the mail.
Prerequisites
This course assumes that you write good code in C/C++ and are familiar with the Linux environment. You will write a lot of C/C++ code running on remote Linux machines, and also use profiling tools to analyze the performance of programs.
Requirements:
- You have taken a C/C++ course or have a fair amount of practical experience with C/C++ programming. For those who are not familiar with C++, you may need to study it by yourself.
- You know how to connect to remote machines with
ssh,scp, etc. - You know how to work on Linux systems (i.e., basic commands such as
cd,rm, and running executables and GNU tools such asmakeandgcc).
Overview
This course examines current research in parallel and cloud computing, with an emphasis on several programming models. Topics covered include
- Parallel and distributed programming for cloud computing
- Introduction to parallel hardware and software
- Shared-memory programming with OpenMP
- Shared-memory programming with Pthreads
- Distributed-memory programming with MPI
- Data parallel programming with CUDA
- Data parallel programming with OpenCL
- Performance benchmarking
Week-by-week Schedule
| Week | Lecture | Slides | Assignments | Peer-review comments |
|---|---|---|---|---|
| 1 (9/17) | Course introduction | L0 | HW0 release | |
| 2 (9/24) | Parallel and distributed programming | L1 | ||
| 3 (10/1) | Moon Festival | |||
| 4 (10/8) | Introduction to parallel hardware and software | L2 | HW1 release | |
| 5 (10/15) | Shared-memory programming with Pthreads | L3-1 | ||
| 6 (10/22) | Shared-memory programming with Pthreads | L3-2 | HW2 release, HW1 due | |
| 7 (10/29) | Shared-memory programming with OpenMP | L4-1 | ||
| 8 (11/05) | Shared-memory programming with OpenMP | L4-2 | HW3 release, HW2 due | |
| 9 (11/12) | Distributed-memory programming with MPI | L5-1 L5-2 | ||
| 10 (11/19) | Data parallel programming with CUDA | L6 | HW4 release, HW3 due | |
| 11 (11/26) | Data parallel programming with CUDA | |||
| 12 (12/03) | Data parallel programming with OpenCL | L7 | ||
| 13 (12/10) | Data parallel programming with OpenCL / OpenACC | L8 | HW5 release, HW4 due | |
| 14 (12/17) | Final project presentations | |||
| 15 (12/24) | Final project presentations | HW6 release, HW5 due | ||
| 16 (12/31) | Final project presentations | |||
| 17 (01/07) | Final project presentations | HW6 due |
Textbook
Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann; 1 edition (January 21, 2011).
Course Project
See here for details.
Grading
Grades will be assigned based on
- Homework assignments (70%) and
- Final project and presentation (30%).
These weights are subject to minor variation.
References
- C++ Programming
- Gerassimos Barlas. 2014. Stanley B. Lippman, Josée Lajoie, Barbara E. Moo.C++ Primer, 5th Edition. ISBN: 9780321714114
- Scott Meyers. 2014. Effective Modern C++. ISBN: 9781491903995
- Parallel Programming.
- Multicore and GPU Programming: An Integrated Approach, 1st Edition. ISBN: 9780124171374
- Benedict R. Gaster et al. 2012. Heterogeneous Computing with OpenCL, 2nd Edition. ISBN: 9780124058941
- David B. Kirk, Wen-mei W.Hwu. 2017. Programming Massively Parallel Processors, 3rd Edition. ISBN: 9780124159921
- Stanford CS149, Fall 2019. Course: Parallel computing. http://cs149.stanford.edu/fall19/
- CMU 15-418/15-618, Fall 2019. Course: Parallel Computer Architecture and Programming. http://www.cs.cmu.edu/afs/cs/academic/class/15418-f19/www
- Profiling
- Brendan Gregg. 2013. Systems Performance: Enterprise and the Cloud, 1st Edition. http://www.brendangregg.com/sysperfbook.html
- Brendan Gregg. 2020. Linux Perf Examples. Retrieved from http://www.brendangregg.com/perf.html
- Anish, Jon, and Jose. 2020. Debugging and Profiling of “The Missing Semester of Your CS Education”. Retrieved from https://missing.csail.mit.edu/2020/debugging-profiling/