Parallel Programming @ NYCU, Spring 2021

Announcement

• Logistics of the presentation sessions is available here.
• Presentation schedule registration is available at https://ppt.cc/fNFLax. Check out the slides for more details.
• Scores of assignment are available at the score sheet.
• Group member registration is now available at the group sheet. Registration is due on March 11.

General Information

Course name: Parallel Programming 平行程式(在職專班)

Class time: Rabc (Thursdays)

Class location: EC015(工程三館)

Facebook group: Link

Refer to the SYLLABUS for more details.

People

Prerequisites

This course assumes that you write good code in C/C++ and are familiar with the Linux environment. You will write a lot of C/C++ code running on remote Linux machines, and also use profiling tools to analyze the performance of programs.

Requirements:

• You have taken a C/C++ course or have a fair amount of practical experience with C/C++ programming. For those who are not familiar with C++, you may need to study it by yourself.
• You know how to connect to remote machines with ssh, scp, etc.
• You know how to work on Linux systems (i.e., basic commands such as cd, rm, and running executables and GNU tools such as make and gcc).

Overview

This course examines current research in parallel and cloud computing, with an emphasis on several programming models. Topics covered include

• Parallel and distributed programming for cloud computing
• Introduction to parallel hardware and software
• Shared-memory programming with OpenMP
• Shared-memory programming with Pthreads
• Distributed-memory programming with MPI
• Data parallel programming with CUDA
• Data parallel programming with OpenCL
• Performance benchmarking

Week-by-week Schedule

Textbook

Peter Pacheco, An Introduction to Parallel Programming, Morgan Kaufmann; 1 edition (January 21, 2011).

Course Project

See the project page for details.

Grading

Grades will be assigned based on

• Homework assignments (70%) and
• Final project and presentation (30%).

These weights are subject to minor variation.

References

• C++ Programming
  1. Gerassimos Barlas. 2014. Stanley B. Lippman, Josée Lajoie, Barbara E. Moo.C++ Primer, 5th Edition. ISBN: 9780321714114
  2. Scott Meyers. 2014. Effective Modern C++. ISBN: 9781491903995
• Parallel Programming.
  1. Multicore and GPU Programming: An Integrated Approach, 1st Edition. ISBN: 9780124171374
  2. Benedict R. Gaster et al. 2012. Heterogeneous Computing with OpenCL, 2nd Edition. ISBN: 9780124058941
  3. David B. Kirk, Wen-mei W.Hwu. 2017. Programming Massively Parallel Processors, 3rd Edition. ISBN: 9780124159921
  4. Stanford CS149, Fall 2019. Course: Parallel computing. http://cs149.stanford.edu/fall19/
  5. CMU 15-418/15-618, Fall 2019. Course: Parallel Computer Architecture and Programming. http://www.cs.cmu.edu/afs/cs/academic/class/15418-f19/www
  6. Jserv. 2021. 並行和多執行緒程式設計. https://hackmd.io/@sysprog/concurrency/
• Profiling
  1. Brendan Gregg. 2013. Systems Performance: Enterprise and the Cloud, 1st Edition. http://www.brendangregg.com/sysperfbook.html
  2. Brendan Gregg. 2020. Linux Perf Examples. Retrieved from http://www.brendangregg.com/perf.html
  3. Anish, Jon, and Jose. 2020. Debugging and Profiling of “The Missing Semester of Your CS Education”. Retrieved from https://missing.csail.mit.edu/2020/debugging-profiling/

History of the Course

• Parallel Programming @ NYCU, Spring 2021 (在職專班)
• Parallel Programming @ NCTU, Fall 2020 (碩士班)