Lawrence Snyder is Professor of Computer Science and Engineering at the University of Washington in Seattle. He received his PhD from Carnegie Mellon University and has devoted most of his career to parallel computation research, including architecture, algorithms and languages. With Calvin Lin and UW graduate students, he developed the ZPL parallel programming language. He is a fellow of the ACM and IEEE. He is an ardent traveler, enthusiastic theater-goer and occasional skier. Calvin Lin is an Associate Professor of Computer Sciences at The University of Texas at Austin, where he also serves as Director of the Turing Scholars Honors Program of undergraduate CS majors. He received his PhD from the University of Washington under the supervision of Lawrence Snyder. His current research interests include compilers and micro-architecture. In his spare time, he is an avid ultimate Frisbee player and coach of UT’s Men’s Ultimate Frisbee team.

• Chapter 1 Introduction: Parallelism = Opportunities + ChallengesThe Power and Potential of ParallelismExamining Sequential and Parallel ProgramsA Paradigm ShiftParallelism Using Multiple Instruction StreamsThe Goals: Scalable Performance and PortabilitySummaryHistorical ContextExercisesChapter 2 Parallel Computers And Their ModelBalancing Machine Specifics with PortabilityA Look at Five Parallel ComputersThe RAM: An Abstraction of a Sequential ComputerThe PRAM: A Parallel Computer ModelThe CTA: A Practical Parallel Computer ModelMemory Reference MechanismsA Closer Look at CommunicationApplying the CTA ModelSummaryHistorical PerspectiveExercisesChapter 3 Reasoning about PerformanceIntroductionMotivation and Some Basic ConceptsSources of Performance LossParallel StructureReasoning about PerformancePerformance Trade-OffsMeasuring PerformanceWhat should we measure?SummaryHistorical PerspectiveExercisesChapter 4 First Steps Towards Parallel ProgrammingTask and Data ParallelismPeril-LCount 3s ExampleConceptualizing ParallelismAlphabetizing ExampleComparison of Three SolutionsSummaryHistorical PerspectiveExercisesChapter 5 Scalable Algorithmic TechniquesThe Inevitability of TreesBlocks of Independent ComputationSchwartz’ AlgorithmAssigning Work To Processes StaticallyAssigning Work to Processes DynamicallyThe Reduce & Scan Abstractions TreesSummaryHistorical ContextExercisesChapter 6 Programming with ThreadsPOSIX ThreadsThread Creation and DestructionMutual ExclusionSynchronizationSafety IssuesPerformance IssuesOpen MPThe Count 3s ExampleSemantic Limitations onReductionThread Behavior and InteractionSectionsSummary of OpenMPJava ThreadsSummaryHistorical PerspectivesExercisesChapter 7 Local View Programming LanguagesMPI: The Message Passing InterfaceGetting StartedSafety IssuesPerformance IssuesCo-Array FortranUnified Parallel CTitaniumSummaryExercisesChapter 8 Global View Programming LanguagesThe Z-level Programming LanguageBasic Concepts of ZPLLife, An ExampleDesign PrinciplesManipulating Arrays Of Different RanksReordering Data With RemapParallel Execution of ZPLPerformance ModelSummaryNESLHistorical ContextExercisesChapter 9 Assessing Our KnowledgeIntroductionEvaluating Existing ApproachesLessons for the FutureSummaryHistorical PerspectivesExercisesChapter 10 Future Directions in Parallel ProgrammingAttached ProcessorsGrid ComputingTransactional MemorySummaryExercisesChapter 11 Capstone Project: Designing a Parallel ProgramIntroductionMotivationGetting StartedSummaryHistorical PerspectiveExercisesAppendix 1 More Advanced Concepts

"...the first basic book on the subject that I've ever seen that seems to have the pulse on the true issues of parallelism that are relevant for students." - Alan Edelman, MIT "Principles of Parallel Programming is a wonderful book and I plan to use it in our new parallel programming course..." - Peiyi Tang, University of Arkansas, Little Rock "I like [Principles of Parallel Programming] very much for a few specific reasons: it's concise, covers the most relevant topics but does not take thousand pages to do it, it is hands on and it covers...recent developments with multi-core and GPGPU." - Edin Hodzic, Santa Clara University

Lawrence Snyder is Professor of Computer Science and Engineering at the University of Washington in Seattle. He received his PhD from Carnegie Mellon University and has devoted most of his career to parallel computation research, including architecture, algorithms and languages. With Calvin Lin and UW graduate students, he developed the ZPL parallel programming language. He is a fellow of the ACM and IEEE. He is an ardent traveler, enthusiastic theater-goer and occasional skier. Calvin Lin is an Associate Professor of Computer Sciences at The University of Texas at Austin, where he also serves as Director of the Turing Scholars Honors Program of undergraduate CS majors. He received his PhD from the University of Washington under the supervision of Lawrence Snyder. His current research interests include compilers and micro-architecture. In his spare time, he is an avid ultimate Frisbee player and coach of UT’s Men’s Ultimate Frisbee team.

• Chapter 1 Introduction: Parallelism = Opportunities + ChallengesThe Power and Potential of ParallelismExamining Sequential and Parallel ProgramsA Paradigm ShiftParallelism Using Multiple Instruction StreamsThe Goals: Scalable Performance and PortabilitySummaryHistorical ContextExercisesChapter 2 Parallel Computers And Their ModelBalancing Machine Specifics with PortabilityA Look at Five Parallel ComputersThe RAM: An Abstraction of a Sequential ComputerThe PRAM: A Parallel Computer ModelThe CTA: A Practical Parallel Computer ModelMemory Reference MechanismsA Closer Look at CommunicationApplying the CTA ModelSummaryHistorical PerspectiveExercisesChapter 3 Reasoning about PerformanceIntroductionMotivation and Some Basic ConceptsSources of Performance LossParallel StructureReasoning about PerformancePerformance Trade-OffsMeasuring PerformanceWhat should we measure?SummaryHistorical PerspectiveExercisesChapter 4 First Steps Towards Parallel ProgrammingTask and Data ParallelismPeril-LCount 3s ExampleConceptualizing ParallelismAlphabetizing ExampleComparison of Three SolutionsSummaryHistorical PerspectiveExercisesChapter 5 Scalable Algorithmic TechniquesThe Inevitability of TreesBlocks of Independent ComputationSchwartz’ AlgorithmAssigning Work To Processes StaticallyAssigning Work to Processes DynamicallyThe Reduce & Scan Abstractions TreesSummaryHistorical ContextExercisesChapter 6 Programming with ThreadsPOSIX ThreadsThread Creation and DestructionMutual ExclusionSynchronizationSafety IssuesPerformance IssuesOpen MPThe Count 3s ExampleSemantic Limitations onReductionThread Behavior and InteractionSectionsSummary of OpenMPJava ThreadsSummaryHistorical PerspectivesExercisesChapter 7 Local View Programming LanguagesMPI: The Message Passing InterfaceGetting StartedSafety IssuesPerformance IssuesCo-Array FortranUnified Parallel CTitaniumSummaryExercisesChapter 8 Global View Programming LanguagesThe Z-level Programming LanguageBasic Concepts of ZPLLife, An ExampleDesign PrinciplesManipulating Arrays Of Different RanksReordering Data With RemapParallel Execution of ZPLPerformance ModelSummaryNESLHistorical ContextExercisesChapter 9 Assessing Our KnowledgeIntroductionEvaluating Existing ApproachesLessons for the FutureSummaryHistorical PerspectivesExercisesChapter 10 Future Directions in Parallel ProgrammingAttached ProcessorsGrid ComputingTransactional MemorySummaryExercisesChapter 11 Capstone Project: Designing a Parallel ProgramIntroductionMotivationGetting StartedSummaryHistorical PerspectiveExercisesAppendix 1 More Advanced Concepts

"...the first basic book on the subject that I've ever seen that seems to have the pulse on the true issues of parallelism that are relevant for students." - Alan Edelman, MIT "Principles of Parallel Programming is a wonderful book and I plan to use it in our new parallel programming course..." - Peiyi Tang, University of Arkansas, Little Rock "I like [Principles of Parallel Programming] very much for a few specific reasons: it's concise, covers the most relevant topics but does not take thousand pages to do it, it is hands on and it covers...recent developments with multi-core and GPGPU." - Edin Hodzic, Santa Clara University

Lawrence Snyder is Professor of Computer Science and Engineering at the University of Washington in Seattle. He received his PhD from Carnegie Mellon University and has devoted most of his career to parallel computation research, including architecture, algorithms and languages. With Calvin Lin and UW graduate students, he developed the ZPL parallel programming language. He is a fellow of the ACM and IEEE. He is an ardent traveler, enthusiastic theater-goer and occasional skier. Calvin Lin is an Associate Professor of Computer Sciences at The University of Texas at Austin, where he also serves as Director of the Turing Scholars Honors Program of undergraduate CS majors. He received his PhD from the University of Washington under the supervision of Lawrence Snyder. His current research interests include compilers and micro-architecture. In his spare time, he is an avid ultimate Frisbee player and coach of UT’s Men’s Ultimate Frisbee team.

• Chapter 1 Introduction: Parallelism = Opportunities + ChallengesThe Power and Potential of ParallelismExamining Sequential and Parallel ProgramsA Paradigm ShiftParallelism Using Multiple Instruction StreamsThe Goals: Scalable Performance and PortabilitySummaryHistorical ContextExercisesChapter 2 Parallel Computers And Their ModelBalancing Machine Specifics with PortabilityA Look at Five Parallel ComputersThe RAM: An Abstraction of a Sequential ComputerThe PRAM: A Parallel Computer ModelThe CTA: A Practical Parallel Computer ModelMemory Reference MechanismsA Closer Look at CommunicationApplying the CTA ModelSummaryHistorical PerspectiveExercisesChapter 3 Reasoning about PerformanceIntroductionMotivation and Some Basic ConceptsSources of Performance LossParallel StructureReasoning about PerformancePerformance Trade-OffsMeasuring PerformanceWhat should we measure?SummaryHistorical PerspectiveExercisesChapter 4 First Steps Towards Parallel ProgrammingTask and Data ParallelismPeril-LCount 3s ExampleConceptualizing ParallelismAlphabetizing ExampleComparison of Three SolutionsSummaryHistorical PerspectiveExercisesChapter 5 Scalable Algorithmic TechniquesThe Inevitability of TreesBlocks of Independent ComputationSchwartz’ AlgorithmAssigning Work To Processes StaticallyAssigning Work to Processes DynamicallyThe Reduce & Scan Abstractions TreesSummaryHistorical ContextExercisesChapter 6 Programming with ThreadsPOSIX ThreadsThread Creation and DestructionMutual ExclusionSynchronizationSafety IssuesPerformance IssuesOpen MPThe Count 3s ExampleSemantic Limitations onReductionThread Behavior and InteractionSectionsSummary of OpenMPJava ThreadsSummaryHistorical PerspectivesExercisesChapter 7 Local View Programming LanguagesMPI: The Message Passing InterfaceGetting StartedSafety IssuesPerformance IssuesCo-Array FortranUnified Parallel CTitaniumSummaryExercisesChapter 8 Global View Programming LanguagesThe Z-level Programming LanguageBasic Concepts of ZPLLife, An ExampleDesign PrinciplesManipulating Arrays Of Different RanksReordering Data With RemapParallel Execution of ZPLPerformance ModelSummaryNESLHistorical ContextExercisesChapter 9 Assessing Our KnowledgeIntroductionEvaluating Existing ApproachesLessons for the FutureSummaryHistorical PerspectivesExercisesChapter 10 Future Directions in Parallel ProgrammingAttached ProcessorsGrid ComputingTransactional MemorySummaryExercisesChapter 11 Capstone Project: Designing a Parallel ProgramIntroductionMotivationGetting StartedSummaryHistorical PerspectiveExercisesAppendix 1 More Advanced Concepts

"...the first basic book on the subject that I've ever seen that seems to have the pulse on the true issues of parallelism that are relevant for students." - Alan Edelman, MIT "Principles of Parallel Programming is a wonderful book and I plan to use it in our new parallel programming course..." - Peiyi Tang, University of Arkansas, Little Rock "I like [Principles of Parallel Programming] very much for a few specific reasons: it's concise, covers the most relevant topics but does not take thousand pages to do it, it is hands on and it covers...recent developments with multi-core and GPGPU." - Edin Hodzic, Santa Clara University