Karama Kanoun is Directeur de Recherche at LAAS-CNRS, France. Her research interests include the modeling and evaluation of computer system dependability. She was the principal investigator for the DBench (Dependability Benchmarking) European project, and has been a consultant for the European Space Agency, Ansaldo Trasporti, and the International Telecommunication Union. Kanoun is vice-chair of the IFIP WG 10.4 on Dependable Computing and Fault Tolerance and chairs its SIG on Dependability Benchmarking. She also chairs the French SEE Technical Committee on Trustworthy Computer Systems. Lisa Spainhower is an IBM Distinguished Engineer in the System Design organization of Systems and Technology Group (STG). STG designs and develops IBM's semiconductor technology, ranging from small x86-based servers to clusters of mainframes, operating systems, and storage subsystems. She is also a member of the IBM Academy of Technology,?IEEE, IEEE Computer Society, and the Technical Committee on Fault-Tolerant Computing Executive Committee. Spainhower is vice-chair of the IFIP WG 10.4 SIG on Dependability Benchmarking.

• Preface vii Contributors xiPrologue: Dependability Benchmarking: A Reality or a Dream? xiiiKarama Kanoun, Phil Koopman, Henrique Madeira, and Lisa Spainhower1 The Autonomic Computing Benchmark 3Joyce Coleman, Tony Lau, Bhushan Lokhande, Peter Shum, Robert Wisniewski, and Mary Peterson Yost2 Analytical Reliability, Availability, and Serviceability Benchmarks 23Richard Elling, Ira Pramanick, James Mauro, William Bryson, and Dong Tang3 System Recovery Benchmarks 35Richard Elling, Ira Pramanick, James Mauro, William Bryson, and Dong Tang4 Dependability Benchmarking Using Environmental Test Tools 55Cristian Constantinescu5 Dependability Benchmark for OLTP Systems 63Marco Vieira, João Durães, and Henrique Madeira6 Dependability Benchmarking of Web Servers 91João Durães, Marco Vieira, and Henrique Madeira7 Dependability Benchmark of Automotive Engine Control Systems 111Juan-Carlos Ruiz, Pedro Gil, Pedro Yuste, and David de-Andrés8 Toward Evaluating the Dependability of Anomaly Detectors 141Kymie M. C. Tan and Roy A. Maxion9 Vajra: Evaluating Byzantine-Fault-Tolerant Distributed Systems 163Sonya J. Wierman and Priya Narasimhan10 User-Relevant Software Reliability Benchmarking 185Mario R. Garzia11 Interface Robustness Testing: Experience and Lessons Learned from the Ballista Project 201Philip Koopman, Kobey DeVale, and John DeVale12 Windows and Linux Robustness Benchmarks with Respect to Application Erroneous Behavior 227Karama Kanoun, Yves Crouzet, Ali Kalakech, and Ana-Elena Rugina13 DeBERT: Dependability Benchmarking of Embedded Real-Time Off-the-Shelf Components for Space Applications 255Diamantino Costa, Ricardo Barbosa, Ricardo Maia, and Francisco Moreira14 Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel 285Arnaud Albinet, Jean Arlat, and Jean-Charles Fabre15 Benchmarking the Operating System against Faults Impacting Operating System Functions 311Ravishankar Iyer, Zbigniew Kalbarczyk, and Weining Gu16 Neutron Soft Error Rate Characterization of Microprocessors 341Cristian ConstantinescuIndex 351

"This is a technical book for sophisticated systems developments and implementers, but it is an excellent read for that audience, on a pressing topic for today's infrastructure planning and design." (Computing Reviews, January 19, 2009)

Karama Kanoun is Directeur de Recherche at LAAS-CNRS, France. Her research interests include the modeling and evaluation of computer system dependability. She was the principal investigator for the DBench (Dependability Benchmarking) European project, and has been a consultant for the European Space Agency, Ansaldo Trasporti, and the International Telecommunication Union. Kanoun is vice-chair of the IFIP WG 10.4 on Dependable Computing and Fault Tolerance and chairs its SIG on Dependability Benchmarking. She also chairs the French SEE Technical Committee on Trustworthy Computer Systems. Lisa Spainhower is an IBM Distinguished Engineer in the System Design organization of Systems and Technology Group (STG). STG designs and develops IBM's semiconductor technology, ranging from small x86-based servers to clusters of mainframes, operating systems, and storage subsystems. She is also a member of the IBM Academy of Technology,?IEEE, IEEE Computer Society, and the Technical Committee on Fault-Tolerant Computing Executive Committee. Spainhower is vice-chair of the IFIP WG 10.4 SIG on Dependability Benchmarking.

• Preface vii Contributors xiPrologue: Dependability Benchmarking: A Reality or a Dream? xiiiKarama Kanoun, Phil Koopman, Henrique Madeira, and Lisa Spainhower1 The Autonomic Computing Benchmark 3Joyce Coleman, Tony Lau, Bhushan Lokhande, Peter Shum, Robert Wisniewski, and Mary Peterson Yost2 Analytical Reliability, Availability, and Serviceability Benchmarks 23Richard Elling, Ira Pramanick, James Mauro, William Bryson, and Dong Tang3 System Recovery Benchmarks 35Richard Elling, Ira Pramanick, James Mauro, William Bryson, and Dong Tang4 Dependability Benchmarking Using Environmental Test Tools 55Cristian Constantinescu5 Dependability Benchmark for OLTP Systems 63Marco Vieira, João Durães, and Henrique Madeira6 Dependability Benchmarking of Web Servers 91João Durães, Marco Vieira, and Henrique Madeira7 Dependability Benchmark of Automotive Engine Control Systems 111Juan-Carlos Ruiz, Pedro Gil, Pedro Yuste, and David de-Andrés8 Toward Evaluating the Dependability of Anomaly Detectors 141Kymie M. C. Tan and Roy A. Maxion9 Vajra: Evaluating Byzantine-Fault-Tolerant Distributed Systems 163Sonya J. Wierman and Priya Narasimhan10 User-Relevant Software Reliability Benchmarking 185Mario R. Garzia11 Interface Robustness Testing: Experience and Lessons Learned from the Ballista Project 201Philip Koopman, Kobey DeVale, and John DeVale12 Windows and Linux Robustness Benchmarks with Respect to Application Erroneous Behavior 227Karama Kanoun, Yves Crouzet, Ali Kalakech, and Ana-Elena Rugina13 DeBERT: Dependability Benchmarking of Embedded Real-Time Off-the-Shelf Components for Space Applications 255Diamantino Costa, Ricardo Barbosa, Ricardo Maia, and Francisco Moreira14 Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel 285Arnaud Albinet, Jean Arlat, and Jean-Charles Fabre15 Benchmarking the Operating System against Faults Impacting Operating System Functions 311Ravishankar Iyer, Zbigniew Kalbarczyk, and Weining Gu16 Neutron Soft Error Rate Characterization of Microprocessors 341Cristian ConstantinescuIndex 351

"This is a technical book for sophisticated systems developments and implementers, but it is an excellent read for that audience, on a pressing topic for today's infrastructure planning and design." (Computing Reviews, January 19, 2009)

Karama Kanoun is Directeur de Recherche at LAAS-CNRS, France. Her research interests include the modeling and evaluation of computer system dependability. She was the principal investigator for the DBench (Dependability Benchmarking) European project, and has been a consultant for the European Space Agency, Ansaldo Trasporti, and the International Telecommunication Union. Kanoun is vice-chair of the IFIP WG 10.4 on Dependable Computing and Fault Tolerance and chairs its SIG on Dependability Benchmarking. She also chairs the French SEE Technical Committee on Trustworthy Computer Systems. Lisa Spainhower is an IBM Distinguished Engineer in the System Design organization of Systems and Technology Group (STG). STG designs and develops IBM's semiconductor technology, ranging from small x86-based servers to clusters of mainframes, operating systems, and storage subsystems. She is also a member of the IBM Academy of Technology,?IEEE, IEEE Computer Society, and the Technical Committee on Fault-Tolerant Computing Executive Committee. Spainhower is vice-chair of the IFIP WG 10.4 SIG on Dependability Benchmarking.

• Preface vii Contributors xiPrologue: Dependability Benchmarking: A Reality or a Dream? xiiiKarama Kanoun, Phil Koopman, Henrique Madeira, and Lisa Spainhower1 The Autonomic Computing Benchmark 3Joyce Coleman, Tony Lau, Bhushan Lokhande, Peter Shum, Robert Wisniewski, and Mary Peterson Yost2 Analytical Reliability, Availability, and Serviceability Benchmarks 23Richard Elling, Ira Pramanick, James Mauro, William Bryson, and Dong Tang3 System Recovery Benchmarks 35Richard Elling, Ira Pramanick, James Mauro, William Bryson, and Dong Tang4 Dependability Benchmarking Using Environmental Test Tools 55Cristian Constantinescu5 Dependability Benchmark for OLTP Systems 63Marco Vieira, João Durães, and Henrique Madeira6 Dependability Benchmarking of Web Servers 91João Durães, Marco Vieira, and Henrique Madeira7 Dependability Benchmark of Automotive Engine Control Systems 111Juan-Carlos Ruiz, Pedro Gil, Pedro Yuste, and David de-Andrés8 Toward Evaluating the Dependability of Anomaly Detectors 141Kymie M. C. Tan and Roy A. Maxion9 Vajra: Evaluating Byzantine-Fault-Tolerant Distributed Systems 163Sonya J. Wierman and Priya Narasimhan10 User-Relevant Software Reliability Benchmarking 185Mario R. Garzia11 Interface Robustness Testing: Experience and Lessons Learned from the Ballista Project 201Philip Koopman, Kobey DeVale, and John DeVale12 Windows and Linux Robustness Benchmarks with Respect to Application Erroneous Behavior 227Karama Kanoun, Yves Crouzet, Ali Kalakech, and Ana-Elena Rugina13 DeBERT: Dependability Benchmarking of Embedded Real-Time Off-the-Shelf Components for Space Applications 255Diamantino Costa, Ricardo Barbosa, Ricardo Maia, and Francisco Moreira14 Benchmarking the Impact of Faulty Drivers: Application to the Linux Kernel 285Arnaud Albinet, Jean Arlat, and Jean-Charles Fabre15 Benchmarking the Operating System against Faults Impacting Operating System Functions 311Ravishankar Iyer, Zbigniew Kalbarczyk, and Weining Gu16 Neutron Soft Error Rate Characterization of Microprocessors 341Cristian ConstantinescuIndex 351

"This is a technical book for sophisticated systems developments and implementers, but it is an excellent read for that audience, on a pressing topic for today's infrastructure planning and design." (Computing Reviews, January 19, 2009)