The Massively Parallel Programming Blog

Parallel Forall is a GPU Computing developer blog, focused on providing detailed technical information on a variety of massively parallel programming topics, including CUDA C/C++, OpenACCGPU-accelerated libraries, Machine Learning, GPU programming techniques, and much more. At Parallel Forall you will find useful information about productive, high-performance programming techniques for the latest GPU technology. Readers have opportunities for discussion with leading experts in GPU computing, and topics featured in Parallel Forall are of interest to programmers of all levels, and from a variety of disciplines.

First and foremost this blog is “For All”; please get in touch (via the contact form) and tell us what topics you’d like to learn about!

Parallel Forall is curated by Mark Harris, Chief Technologist for GPU Computing Software at NVIDIA. Mark has over ten years of experience developing software for GPUs, ranging from graphics and games, to physically based simulation, to parallel algorithms and high-performance computing. Mark has used GPUs for general-purpose computing since before they were up for the task—before they even supported floating point arithmetic! As a Ph.D. student at UNC  he developed real-time cloud simulation and rendering software for GPUs (simulating clouds, not simulation in “the cloud”!). In 2002 Mark recognized a nascent trend in computing and coined a name for it: GPGPU (General-Purpose computing on Graphics Processing Units), and started to provide a forum for those working in the field to share and discuss their work. has grown into a top destination for information about GPU computing.

You can follow @harrism on Twitter, or email “first initial last name at”.

Today, GPU computing truly is a mainstream technology. World-changing science problems are being solved on GPUs. Many of the Top500 fastest supercomputers in the world rely on GPUs. GPUs can be programmed using the most popular programming languages, and there are dozens of open-source and commercial GPU-accelerated libraries available. GPU computing today is a growing industry with a mature ecosystem of applications, tools, and service providers.