Parallel Computing Architectures and APIs

Vivek Kale has more than two decades of professional IT experience during which he has handled and consulted on various aspects of enterprise-wide information modeling, enterprise architectures, business process re-design, and, e-business architectures. He has been Group CIO of Essar Group, the steel/oil & gas major of India, as well as, Raymond Ltd., the textile & apparel major of India. He is a seasoned practitioner in enhancing business agility through digital transformation of business models, enterprise architecture and business processes, and, enhancing IT-enabled enterprise intelligence (EQ). He has authored books on Cloud Computing and Big Data Computing. He is also author of Big Data Computing: A Guide for Business and Technology Managers(CRC Press, 2016), Agile Network Businesses: Collaboration, Coordination, and Competitive Advantage (CRC Press 2017), and, Digital Transformation of Enterprise Architecture (CRC Press 2020).

• Preface..............................................................................................................................................xvAcknowledgments....................................................................................................................... xxiAuthor..........................................................................................................................................xxiii1 Uniprocessor Computers........................................................................................................12 Processor Physics and Moore’s Law.................................................................................. 17Section I Genesis of Parallel Computing3 Processor Basics..................................................................................................................... 374 Networking Basics................................................................................................................ 515 Distributed Systems Basics.................................................................................................65Section II Road to Parallel Computing6 Parallel Systems.....................................................................................................................837 Parallel Computing Models................................................................................................998 Parallel Algorithms............................................................................................................. 115Section III Parallel Computing Architectures9 Parallel Computing Architecture Basics......................................................................... 13110 Shared Memory Architecture........................................................................................... 14111 Message-Passing Architecture.......................................................................................... 15712 Stream Processing Architecture....................................................................................... 167Section IV Parallel Computing Programming13 Parallel Computing Programming Basics...................................................................... 18114 Shared-memory Parallel Programming with OpenMP.............................................. 21315 Message Passing Parallel Programming with MPI......................................................22316 Stream Processing Programming with CUDA, OpenCL, and OpenACC............... 239Section V Internet of Things Big Data Stream Processing17 Internet of Things (IoT) Technologies............................................................................25518 Sensor Data Processing...................................................................................................... 28719 Big Data Computing........................................................................................................... 31320 Big Data Stream Processing..............................................................................................335Epilogue: Quantum Computing................................................................................................ 361References.....................................................................................................................................365Index.............................................................................................................................................. 367