What are Your Favorite Parallel Programming References?

Recently a colleague asked if I could recommend a good parallel programming textbook. Since it isn’t the first time I’ve been asked this question, and it comes up from time to time, I thought it would be interesting to share the question with you, dear reader.

What are your favorite books about parallel programming (or parallel computing)? While we’re at it, why stop at books? What are your favorite parallel programming papers, journals, or magazines? Where do you look online to learn about parallel programming (websites, blogs, podcasts, etc)?

Maybe I shouldn’t admit this, but I don’t actually have a general parallel computing textbook on my shelf! Many graduate level parallel computing courses don’t require a specific textbook, instead requiring reading from a variety of sources. The course I took in graduate school took this approach (this course is a recent version; you may want to look at the reading list). I still sometimes refer to the PRAM Algorithms handout, written by Professors Prins and Chatterjee, that we used in the course.  I would recommend it to anyone interested in a theoretical algorithms background for their CUDA programming, since the per-thread-block CUDA programming model is not all that different from the CRCW PRAM model. Continue reading