Introduction to High Performance Scientific Computing

Based on a course developed by the author, Introduction to High Performance Scientific Computing introduces methods for adding parallelism to numerical methods for solving differential equations. It contains exercises and programming projects that facilitate learning as well as examples and discussions based on the C programming language, with additional comments for those already familiar with C . The text provides an overview of concepts and algorithmic techniques for modern scientific computing and is divided into six self-contained parts that can be assembled in any order to create an introductory course using available computer hardware.Part I introduces the C programming language for those not already familiar with programming in a compiled language. Part II describes parallelism on shared memory architectures using OpenMP. Part III details parallelism on computer clusters using MPI for coordinating a computation. Part IV demonstrates the use of graphical programming units (GPUs) to solve problems using the CUDA language for NVIDIA graphics cards. Part V addresses programming on GPUs for non-NVIDIA graphics cards using the OpenCL framework. Finally, Part VI contains a brief discussion of numerical methods and applications, giving the reader an opportunity to test the methods on typical computing problems. Introduction to High Performance Scientific Computing is intended for advanced undergraduate or beginning graduate students who have limited exposure to programming or parallel programming concepts. Extensive knowledge of numerical methods is not assumed. The material can be adapted to the available computational hardware, from OpenMP on simple laptops or desktops to MPI on computer clusters or CUDA and OpenCL for computers containing NVIDIA or other graphics cards. Experienced programmers unfamiliar with parallel programming will benefit from comparing the various methods to determine the type of parallel programming best suited for their application. The book can be used for courses on parallel scientific computing, high performance computing, and numerical methods for parallel computing.