Data Bonanza

MALCOLM ATKINSON, PhD, is Professor of e-Science in the School of Informatics at the University of Edinburgh in Scotland. He is also Data-Intensive Research Group leader, Director of the e-Science Institute, IT architect for the ADMIRE and VERCE EU projects and UK e-Science Envoy. Professor Atkinson has been leading research projects for several decades and served on many advisory bodies.

• CONTRIBUTORS xvFOREWORD xviiPREFACE xixTHE EDITORS xxixPART I STRATEGIES FOR SUCCESS IN THE DIGITAL-DATA REVOLUTION 11. The Digital-Data Challenge 5Malcolm Atkinson and Mark Parsons1.1 The Digital Revolution 51.2 Changing How We Think and Behave 61.3 Moving Adroitly in this Fast-Changing Field 81.4 Digital-Data Challenges Exist Everywhere 81.5 Changing How We Work 91.6 Divide and Conquer Offers the Solution 101.7 Engineering Data-to-Knowledge Highways 122. The Digital-Data Revolution 15Malcolm Atkinson2.1 Data, Information, and Knowledge 162.2 Increasing Volumes and Diversity of Data 182.3 Changing the Ways We Work with Data 283. The Data-Intensive Survival Guide 37Malcolm Atkinson3.1 Introduction: Challenges and Strategy 383.2 Three Categories of Expert 393.3 The Data-Intensive Architecture 413.4 An Operational Data-Intensive System 423.5 Introducing DISPEL 443.6 A Simple DISPEL Example 453.7 Supporting Data-Intensive Experts 473.8 DISPEL in the Context of Contemporary Systems 483.9 Datascopes 513.10 Ramps for Incremental Engagement 543.11 Readers’ Guide to the Rest of This Book 564. Data-Intensive Thinking with DISPEL 61Malcolm Atkinson4.1 Processing Elements 624.2 Connections 644.3 Data Streams and Structure 654.4 Functions 664.5 The Three-Level Type System 724.6 Registry, Libraries, and Descriptions 814.7 Achieving Data-Intensive Performance 864.8 Reliability and Control 1084.9 The Data-to-Knowledge Highway 116PART II DATA-INTENSIVE KNOWLEDGE DISCOVERY 1235. Data-Intensive Analysis 127Oscar Corcho and Jano van Hemert5.1 Knowledge Discovery in Telco Inc. 1285.2 Understanding Customers to Prevent Churn 1305.3 Preventing Churn Across Multiple Companies 1345.4 Understanding Customers by Combining Heterogeneous Public and Private Data 1375.5 Conclusions 1446. Problem Solving in Data-Intensive Knowledge Discovery 147Oscar Corcho and Jano van Hemert6.1 The Conventional Life Cycle of Knowledge Discovery 1486.2 Knowledge Discovery Over Heterogeneous Data Sources 1556.3 Knowledge Discovery from Private and Public, Structured and Nonstructured Data 1586.4 Conclusions 1627. Data-Intensive Components and Usage Patterns 165Oscar Corcho7.1 Data Source Access and Transformation Components 1667.2 Data Integration Components 1727.3 Data Preparation and Processing Components 1737.4 Data-Mining Components 1747.5 Visualization and Knowledge Delivery Components 1768. Sharing and Reuse in Knowledge Discovery 181Oscar Corcho8.1 Strategies for Sharing and Reuse 1828.2 Data Analysis Ontologies for Data Analysis Experts 1858.3 Generic Ontologies for Metadata Generation 1888.4 Domain Ontologies for Domain Experts 1898.5 Conclusions 190PART III DATA-INTENSIVE ENGINEERING 1939. Platforms for Data-Intensive Analysis 197David Snelling9.1 The Hourglass Reprise 1989.2 The Motivation for a Platform 2009.3 Realization 20110. Definition of the DISPEL Language 203Paul Martin and Gagarine Yaikhom10.1 A Simple Example 20410.2 Processing Elements 20510.3 Data Streams 21310.4 Type System 21710.5 Registration 22210.6 Packaging 22410.7 Workflow Submission 22510.8 Examples of DISPEL 22710.9 Summary 23511. DISPEL Development 237Adrian Mouat and David Snelling11.1 The Development Landscape 23711.2 Data-Intensive Workbenches 23911.3 Data-Intensive Component Libraries 24711.4 Summary 24812. DISPEL Enactment 251Chee Sun Liew, Amrey Krause, and David Snelling12.1 Overview of DISPEL Enactment 25112.2 DISPEL Language Processing 25312.3 DISPEL Optimization 25512.4 DISPEL Deployment 26612.5 DISPEL Execution and Control 268PART IV DATA-INTENSIVE APPLICATION EXPERIENCE 27513. The Application Foundations of DISPEL 277Rob Baxter13.1 Characteristics of Data-Intensive Applications 27713.2 Evaluating Application Performance 28013.3 Reviewing the Data-Intensive Strategy 28314. Analytical Platform for Customer Relationship Management 287Maciej Jarka and Mark Parsons14.1 Data Analysis in the Telecoms Business 28814.2 Analytical Customer Relationship Management 28914.3 Scenario 1: Churn Prediction 29114.4 Scenario 2: Cross Selling 29314.5 Exploiting the Models and Rules 29614.6 Summary: Lessons Learned 29915. Environmental Risk Management 301Ladislav Hluchy, Ondrej Habala, Viet Tran, and Branislav Simo15.1 Environmental Modeling 30215.2 Cascading Simulation Models 30315.3 Environmental Data Sources and Their Management 30515.4 Scenario 1: ORAVA 30915.5 Scenario 2: RADAR 31315.6 Scenario 3: SVP 31815.7 New Technologies for Environmental Data Mining 32115.8 Summary: Lessons Learned 32316. Analyzing Gene Expression Imaging Data in Developmental Biology 327Liangxiu Han, Jano van Hemert, Ian Overton, Paolo Besana, and Richard Baldock16.1 Understanding Biological Function 32816.2 Gene Image Annotation 33016.3 Automated Annotation of Gene Expression Images 33116.4 Exploitation and Future Work 34116.5 Summary 34517. Data-Intensive Seismology: Research Horizons 353Michelle Galea, Andreas Rietbrock, Alessandro Spinuso, and Luca Trani17.1 Introduction 35417.2 Seismic Ambient Noise Processing 35617.3 Solution Implementation 35817.4 Evaluation 36917.5 Further Work 37217.6 Conclusions 373PART V DATA-INTENSIVE BEACONS OF SUCCESS 37718. Data-Intensive Methods in Astronomy 381Thomas D. Kitching, Robert G. Mann, Laura E. Valkonen, Mark S. Holliman, Alastair Hume, and Keith T. Noddle18.1 Introduction 38118.2 The Virtual Observatory 38218.3 Data-Intensive Photometric Classification of Quasars 38318.4 Probing the Dark Universe with Weak Gravitational Lensing 38718.5 Future Research Issues 39218.6 Conclusions 39219. The World at One's Fingertips: Interactive Interpretation of Environmental Data 395Jon Blower, Keith Haines, and Alastair Gemmell19.1 Introduction 39519.2 The Current State of the Art 39719.3 The Technical Landscape 40119.4 Interactive Visualization 40319.5 From Visualization to Intercomparison 40619.6 Future Development: The Environmental Cloud 40919.7 Conclusions 41120. Data-Driven Research in the Humanities—the DARIAH Research Infrastructure 417Andreas Aschenbrenner, Tobias Blanke, Christiane Fritze, andWolfgang Pempe20.1 Introduction 41720.2 The Tradition of Digital Humanities 42020.3 Humanities Research Data 42220.4 Use Case 42620.5 Conclusion and Future Development 42921. Analysis of Large and Complex Engineering and Transport Data 431Jim Austin21.1 Introduction 43121.2 Applications and Challenges 43221.3 The Methods Used 43421.4 Future Developments 43821.5 Conclusions 439References 44022. Estimating Species Distributions—Across Space, Through Time, and with Features of the Environment 441Steve Kelling, Daniel Fink, Wesley Hochachka, Ken Rosenberg, Robert Cook, Theodoros Damoulas, Claudio Silva, and William Michener22.1 Introduction 44222.2 Data Discovery, Access, and Synthesis 44322.3 Model Development 44822.4 Managing Computational Requirements 44922.5 Exploring and Visualizing Model Results 45022.6 Analysis Results 45222.7 Conclusion 454PART VI THE DATA-INTENSIVE FUTURE 45923. Data-Intensive Trends 461Malcolm Atkinson and Paolo Besana23.1 Reprise 46123.2 Data-Intensive Applications 46924. Data-Rich Futures 477Malcolm Atkinson24.1 Future Data Infrastructure 47824.2 Future Data Economy 48524.3 Future Data Society and Professionalism 489References 494Appendix A: Glossary 499Michelle Galea and Malcolm AtkinsonAppendix B: DISPEL Reference Manual 507Paul MartinAppendix C: Component Definitions 531Malcolm Atkinson and Chee Sun LiewINDEX 537