Open Textbook:

Programming on Parallel Machines; GPU, Multicore, Clusters and More

Professor Norm Matloff , University of California, Davis

OVERVIEW:

"Why is this book different from all other parallel programming books?"

Suitable for either students or professionals.

Practical viewpoint: There is very little theoretical analysis of parallel algorithms, such as O() analysis, maximum theoretical speedup, acyclic graphs and so on. Extensive coverage of "wizardry" aspects, i.e. material known to experienced practitioners but generally not in books, such as coverage of loop iteration scheduling, memory effects of storing large arrays and so on. Appendices cover systems background, crucial in applied work but always just "assumed" to be knowledge possessed by the readers. Considerable attention is paid to techniques for debugging.

Uses the main parallel platforms---OpenMP, CUDA and MPI---rather than languages that at this stage are largely experimental, such as the elegant-but-not-yet-mainstream Cilk.

Starts with real parallel code right away in Chapter 1, with examples from pthreads, OpenMP and MPI.

Constantly evolving: Like all my open source textbooks, this one is constantly evolving. I continue to add new topics, new examples, more timing analyses, and so on, and of course fix bugs and improve the exposition.

Prerequisites: The student must be easonably adept in programming, and have math background through linear algebra. (An appendix to the book reviews the parts of the latter needed for this book.)

LICENSING:



This work is licensed under a Creative Commons Attribution-No Derivative Works 3.0 United States License. Copyright is retained by N. Matloff in all non-U.S. jurisdictions, but permission to use these materials in teaching is still granted, provided the authorship and licensing information here is displayed. I would appreciate being notified if you use this book for teaching, just so that I know the materials are being put to use, but this is not required.

The book is freely available, subject to the conditions above, and can be downloaded from http://heather.cs.ucdavis.edu/~matloff/158/PLN/ParProcBook.pdf.

AUTHOR'S BIO:

Norm Matloff is a Professor of Computer Science at the University of California, Davis. He was formerly a statistics professor at that university. His research has included a number of diverse areas, and he has been a recipient of the university's Distinguished Teaching Award. He was born and raised in Los Angeles, and earned his PhD in theoretical mathematics (probability theory/functional analysis) at UCLA.