Practical Introduction to Human-in-the-Loop Cyber-Physical Systems

David Nunes, PhD is an information systems advisor and software developer at the Central Portuguese Bank. He holds a Masters degree in Bioinformatics, and a PhD in Informatics, both from the University of Coimbra, Portugal. Jorge Sá Silva, PhD is an Assistant Professor in the Department of Informatics Engineering at the tUniversity of Coimbra, Portugal. Fernando Boavida, PhD is a Professor in the Department of Informatics Engineering at the University of Coimbra and founder of the University's Laboratory of Communications and Telematics of DEI.

• List of Figures xiList of Tables xviiForeword xixPreface xxiAcknowledgments xxiiiList of Abbreviations xxvAbout the CompanionWebsite xxvii1 Introduction 11.1 The Rise of Cyber-Physical Systems 11.2 Humans as Elements of Cyber-Physical Systems 41.3 Objectives and Structure 7Part I Evolution and Theory 92 Evolution of HiTL Technologies 112.1 "Things", Sensors, and the RealWorld 112.2 Human Sensing and Virtual Communities 172.3 In Summary... 233 TheoryofHiTLCPSs 253.1 Taxonomies for HiTLCPSs 253.2 Data Acquisition 283.2.1 Humans as Sets of Sensors 283.2.2 Humans as Communication Nodes 293.3 State Inference 303.3.1 Human Nature 303.3.2 Humans as Processing Nodes 313.4 Actuation 313.4.1 Humans and Robots as Actuators 313.5 In Summary... 324 HITL Technologies and Applications 354.1 Technologies for Supporting HiTLCPS 354.1.1 Data Acquisition 354.1.2 State Inference 394.1.3 Actuation 424.2 Experimental Projects 454.2.1 HiTL in Industry and at Home 464.2.2 HiTL in Healthcare 484.2.3 HiTL in Smartphones and Social Networking 514.3 In Summary... 56Part II Human-in-the-Loop: Hands-On 575 ASampleApp 595.1 A Sample Behavior Change Intervention App 595.2 The Sample App’s Base Architecture 605.2.1 The Android App 605.2.2 The Server 655.3 Enhancing the Sample App with HiTL Emotion-awareness 665.3.1 Choosing a Machine Learning Technique 675.3.2 Implementing Emotion-awareness 685.4 In Summary... 716 Setting up the Development Environment 736.1 Installing Android Studio 736.2 Cloning the Android Project 776.3 Deploying the Server 856.3.1 Installing the Software and Cloning the Server’s Project 856.3.2 Obtaining a Foursquare®’s Client ID and Client Secret 886.3.3 Setting up the Database 916.3.4 Deploying the Server on Tomcat 7 956.4 Testing the Sample App 996.5 In Summary... 1017 Data Acquisition 1037.1 Creating the EmotionTasker 1037.2 Processing Sensory Data 1107.3 In Summary... 1188 StateInference 1218.1 Implementing a Neural Network 1218.2 Requesting User Feedback 1258.2.1 Creating the EmotionFeedback Activity 1258.2.2 Implementing the EmotionSpace View 1278.2.3 Finishing EmotionFeedback 1518.2.4 Showing a Feedback Request Notification 1588.3 Processing User Feedback 1688.3.1 Processing Feedback on the EmotionTasker 1688.3.2 Training the Neural Network 1708.3.3 Sending Emotional Information to the Server 1738.4 In Summary... 1769 Actuation 1799.1 Handling Emotions on the Server 1799.1.1 Parsing JSON Requests 1809.1.2 Creating theWeb Interface 1899.1.3 Creating the Server’s BackgroundThread 1909.1.4 Processing Incoming Emotions 1929.1.5 Pruning Outdated Emotions 2089.2 Finishing up EmotionTasker 2109.2.1 Handling ANN Output 2129.2.2 Posting New Emotion Inferences 2179.3 Providing Positive Reinforcement 2269.3.1 Creating a Motivational Dialog Box 2269.3.2 Enabling the Emotion Heatmaps 2349.4 In Summary… 237Part III Future of Human-In-the-Loop Cyber-Physical Systems 23910 Requirements and Challenges for HiTL Applications 24110.1 Resilience 24110.2 Security and Privacy 24210.3 Standard Communications 24410.4 Localization 24810.5 State Inference 24810.6 Safety 25010.7 In Summary… 25011 Human-in-the-Loop Constraints 25311.1 Technical Limitations 25311.2 Ethical limitations 256A EmotionTasker’s full code 261References 275Index 289