Art and Science of HDR Imaging

John J. McCann, Consultant, McCann Imaging, USAJohn McCann received a B.A. degree in Biology from Harvard University in 1964. He worked in, and later managed, the Vision Research Laboratory at Polaroid from 1961 to 1996. He has studied human color vision, digital image processing, large format instant photography and the reproduction of fine art. His 120 publications have studied Retinex theory, color from rod/Lcone interactions at low light levels, appearance with scattered light, and HDR imaging.He has been a Fellow of the Society of Imaging Science and Technology (IS&T) since 1983. He is a past President of IS&T and the Artists Foundation, Boston. In 1996 he received the SID Certificate of Commendation. He is the IS&T/OSA 2002 Edwin H. Land Medalist, and IS&T 2005 Honorary Member, and is a 2008 Fellow of the Optical Society of America. He is currently consulting and continuing his research on color vision.Alessandro Rizzi, Università degli Studi di Milano, ItalyProfessor Alessandro Rizzi holds a degree in Computer Science at University of Milano and received a PhD in Information Engineering at University of Brescia (Italy). He taught Information Systems and Computer Graphics at University of Brescia and at Politecnico di Milano. He is currently an  assistant professor teaching Multimedia and Human-Computer Interaction, and senior research fellow at the Department of Information Technologies at University of Milano. Since 1990 he has researched in the field of digital imaging and vision. His main research topic is the use of color information in digital images with particular attention to color perception mechanisms. He is the coordinator of the Italian Color GroupConference Chair of Color Conference at IS&T/SPIE Electronic Imaging, and a principle organizer of European Marie Curie Project CREATE.

• About the Authors xix Preface xxiSeries Preface xxiiiAcknowledgements xxvSection A HISTORY OF HDR IMAGING 11 HDR Imaging 31.1 Topics 31.2 Introduction 31.3 Replicas and Reproductions 41.4 A Choice of Metaphors for HDR Reproduction 51.5 Reproduction of Scene Dynamic Range 71.6 HDR Disciplines 81.7 Outline of the Text 101.8 Summary 111.9 References 122 HDR Tools and Defi nitions 132.1 Topics 132.2 Introduction 132.3 Pixels 142.4 Dynamic Ranges 142.5 Measuring Light 172.6 Measuring Color Spaces 182.7 Image Reproduction 212.8 Contrast 242.9 Digital Imaging 252.10 Summary 252.11 References 263 HDR in Natural Scenes 273.1 Topics 273.2 Appearance in HDR and Color Constancy 273.3 Summary 303.4 References 314 HDR in Painting 334.1 Topics 334.2 Introduction 334.3 Ancient Painting 334.4 Perspective 354.5 Chiaroscuro 374.6 Gerritt van Honthorst (Gherardo delle Notti) 374.7 Rembrandt van Vijn 404.8 John Constable 404.9 John Martin 404.10 Impressionism 414.11 Photorealism 434.12 Summary 434.13 References 445 HDR in Film Photography 455.1 Topics 455.2 Introduction 455.3 Multiple Exposures in the 1850s 465.4 HP Robinson 475.5 Hurter and Driffi eld-Scientifi c Calibration of AgX Film Sensitivity 485.6 Sheppard and Mees 505.7 19th Century – Professional Amateur Photography 505.8 20th Century – Corporate Photography 505.9 20th Century Control of Dynamic Range 515.10 Other Silver-Halide Stories 565.11 Summary 565.12 References 576 The Ansel Adams Zone System 596.1 Topics 596.2 Introduction 596.3 Compressing the HDR World into the LDR Print 596.4 Visualization 606.5 Scene Capture 616.6 “Performing the Score” 656.7 Moonrise, Hernandez 666.8 Apparent vs. Physical Contrast 666.9 Summary 666.10 References 687 Electronic HDR Image Processing: Analog and Digital 697.1 Topics 697.2 Introduction 697.3 Human Spatial Vision 697.4 Electronic HDR Image Processing 707.5 Summary 747.6 References 758 HDR and the World of Computer Graphics 778.1 Topics 778.2 Introduction 778.3 Early Years: the 60s 788.4 Early Digital Image Synthesis: the 70s 788.5 The Turning Point: the 80s 798.6 Computational Photorealism: from the 90s 808.7 Summary 808.8 References 819 Review of HDR History 839.1 Topics 839.2 Summary of Disciplines 839.3 Review 849.4 Summary 879.5 References 87Section B MEASURED DYNAMIC RANGES 8910 Actual Dynamic Ranges 9110.1 Topics 9110.2 Introduction 9110.3 Dynamic Range of Light Sensors 9210.4 Bits per Pixel 9310.5 Dynamic Range of Display Devices 9410.6 Interactions of Pixels in Images 9510.7 Summary 9610.8 References 9611 Limits of HDR Scene Capture 9911.1 Topics 9911.2 Introduction 9911.3 HDR Test Targets 9911.4 Camera Veiling Glare Limits 10111.5 Glare in Film Cameras 10711.6 Review 11111.7 Summary 11111.8 References 11212 Limits of HDR in Humans 11312.1 Topics 11312.2 Introduction 11312.3 Visual Appearance of HDR Displays 11312.4 von Honthorst’s Painting and the 4scaleBlack HDR Target 11612.5 HDR Displays and Black and White Mondrian 11612.6 HDR and Tone Scale Maps 11712.7 HDR Displays and Contrast 11712.8 Summary 11712.9 References 11813 Why Does HDR Improve Images? 11913.1 Topics 11913.2 Introduction 11913.3 Why are HDR Images Better? 12013.4 Are Multiple Exposures Necessary? 12013.5 Summary 12113.6 References 121Section C SEPARATING GLARE AND CONTRAST 12314 Two Counteracting Mechanisms: Glare and Contrast 12514.1 Topics 12514.2 Introduction 12514.3 Two Spatial Mechanisms 12614.4 Calculated Retinal Image 12614.5 Measuring the Range of HDR Appearances 13114.6 Calculating the Retinal Image 13114.7 Visualizing the Retinal Image 13114.8 HDR and Uniform Color Space 13214.9 Summary 13214.10 References 13215 Measuring the Range of HDR Appearances 13515.1 Topics 13515.2 Introduction 13515.3 Design of Appearance Scale Target 13615.4 Magnitude Estimation Experiments 13815.5 Scene Dependent Tone Scale 14115.6 Glare and Contrast 14215.7 Summary 14315.8 References 14316 Calculating the Retinal Image 14516.1 Topics 14516.2 Introduction 14516.3 Converting Scene Luminance to Retinal Contrast 14616.4 Calculating Retinal Radiance 14616.5 Changes in the Retinal Image from Glare 14916.6 Appearance and Retinal Image 14916.7 Scene Content and Psychometric Functions 15116.8 Summary 15116.9 References 15217 Visualizing HDR Images 15317.1 Topics 15317.2 Introduction 15317.3 Calculated Retinal Image Contrast 15417.4 Retinal Image Contrast 15517.5 Summary 15917.6 References 15918 HDR and Uniform Color Spaces 16118.1 Topics 16118.2 Introduction 16118.3 Uniform Color Spaces – Psychophysics 16118.4 Color Vision – Physiology 16418.5 Accurate Transformations from CMF to UCS 16518.6 Summary 16718.7 References 16819 Glare: A Major Part of Vision Theory 16919.1 Topics 16919.2 Introduction 16919.3 Glare: Distorts Lightness below Middle Gray, More or Less 16919.4 Pixel-based HDR Image Processing 17019.5 Summary 17119.6 References 171Section D SCENE CONTENT CONTROLS APPEARANCE 17320 Scene Dependent Appearance of Quanta Catch 17520.1 Topics 17520.2 Introduction 17520.3 Models of Vision – A Choice of Paradigms 17520.4 Illumination, Constancy and Surround 17620.5 Maximum’s Enclosure and Distance 17620.6 Size of Maxima 17720.7 Assimilation 17720.8 Maxima and Contrast with Maxima 17721 Illumination, Constancy and Surround 17921.1 Topics 17921.2 Introduction 17921.3 Hipparchus of Nicea 18021.4 Flat-2-D Transparent Displays 18221.5 A Simple Two-Step Physical Description 18321.6 Complex 3-D Scenes 18521.7 Local Maxima 18921.8 Review 19021.9 Summary 19021.10 References 19122 Maximum’s Enclosure and Separation 19322.1 Topics 19322.2 Introduction 19322.3 Experimental Design 19422.4 Lightness Matches – Light Gray on Black 19422.5 Lightness Matches – Dark Gray on Black 19522.6 Dark Gray on Black: Varying White’s Position 19722.7 Review 19822.8 Summary 19922.9 References 20023 Maxima Size and Distribution 20123.1 Topics 20123.2 Introduction 20123.3 Experimental Procedure 20223.4 Controls 20223.5 Dispersion of White (“Snow”) 20223.6 Sides and Corners 20323.7 Lines 20423.8 Equivalent Backgrounds 20523.9 Equivalent Backgrounds and Models of Vision 20723.10 Summary 20723.11 References 20824 From Contrast to Assimilation 20924.1 Topics 20924.2 Introduction 20924.3 Segmented Surrounds 21024.4 Checkerboard Variants 21524.5 Summary 21624.6 References 21625 Maxima and Contrast with Maxima 21725.1 Topics 21725.2 Merger of Aperture and Object Modes 21725.3 Infl uence of the Maxima 21825.4 Summary 219Section E COLOR HDR 22126 HDR, Constancy and Spatial Content 22326.1 Topics 22326.2 Introduction 22326.3 Red and White Projections 22426.4 Color Mondrians 22526.5 Constancy’s On/Off Switch 22526.6 Color of 3-D Mondrians – LDR/HDR Illumination 22626.7 Color Constancy is HDR 22626.8 References 22627 Color Mondrians 22727.1 Topics 22727.2 Introduction 22727.3 Color Mondrians 22927.4 The Signature of Color Constancy 23727.5 Search for Evidence of Adaptation – Averages 24027.6 Transparency in Mondrians 24327.7 Color Assimilation 24327.8 Summary 24427.9 References 24528 Constancy’s On/Off Switch 24728.1 Topics 24728.2 Introduction 24728.3 Maximov’s Shoe Boxes 24728.4 New Maxima Restores Constancy 25028.5 Independent L, M, S Spatial Processing 25128.6 Model Predictions 25328.7 Center-Surround Target – Results 25328.8 Summary 25528.9 References 25629 HDR and 3-D Mondrians 25729.1 Topics 25729.2 Color Constancy and Appearance 25729.3 Color Constancy Models 25829.4 Measuring Changes in Appearance from Changes in Illumination 25929.5 Magnitude Estimation Appearance Measurements 26229.6 Watercolor Rendition Measurements of Appearance 26329.7 Review of 3-D Mondrian Psychophysical Measurements 26629.8 Color Constancy Models 26829.9 Conclusions 27029.10 References 27130 Color Constancy is HDR 27330.1 Topics 27330.2 Introduction 27330.3 Rod Receptors and HDR 27430.4 Assembling Appearance: Color Constancy, Rod Vision and HDR 27930.5 Summary 28030.6 References 280Section F HDR IMAGE PROCESSING 28331 HDR Pixel and Spatial Algorithms 28531.1 Topics 28531.2 Introduction – HDR Image Processing Algorithms 28531.3 One Pixel – Tone Scale Curves 28631.4 Some of the Pixels – Local Processing 28831.5 All of the Pixels 28931.6 All Pixels and Scene Dependent – The Retinex Extended Family 28931.7 Retinex Algorithms 29031.8 ACE Algorithms 29031.9 Analytical, Computational and Variational Algorithms 29031.10 Techniques for Analyzing HDR Algorithms 29031.11 The HDR Story 29131.12 References 29132 Retinex Algorithms 29332.1 Topics 29332.2 Introduction 29332.3 How to Calculate Lightness Using Ratio-Products 29732.4 A Variety of Processing Networks 30132.5 Image Content 30232.6 Real Images – 1975 30732.7 The Extended Family of Retinex Models 31932.8 Algorithm’s Goal 33432.9 References 33733 ACE Algorithms 34133.1 Topics 34133.2 Introduction 34133.3 ACE Algorithm 34133.4 Retinex and ACE 34433.5 ACE Characteristics 34533.6 RACE 34933.7 Other Vision-based Models 35033.8 Summary 35033.9 References 35134 Analytical, Computational and Variational Algorithms 35334.1 Topics 35334.2 Introduction 35334.3 Math in the Framework of the Human Visual System 35434.4 Analytical Retinex Formulas 35434.5 Computational Retinex in Wavelets 35434.6 Retinex and the Variational Techniques 35534.7 Summary 35634.8 References 35735 Evaluation of HDR Algorithms 35935.1 Topics 35935.2 Introduction 35935.3 Quantitative Approaches to Algorithm Evaluation 36035.4 Lightness Test Targets 36135.5 Ratio Metric 36235.6 Quantitative Evaluation of 3-D Mondrians 36735.7 Locality Test Targets 36935.8 Summary 37035.9 Lessons From Quantitative Studies of HDR in Cameras 37135.10 References 37136 The HDR Story 37336.1 Topic 37336.2 Straightforward Technology Stories 37336.3 The HDR Story is Defi ned by Limits 37336.4 HDR Works Well 37436.5 References 375Glossary 377Author Index 385Subject Index 387

“Overall, this book provides an excellent overview of the history of imaging, HDR imaging algorithms, and the abilities of the human visual system. The book is a great achievement for the authors, and it will be well appreciated by anyone who enjoys learning about a field from the key players. Most importantly, it will encourage the reader to think about how visual processing works, and how that process can serve as a model for imaging systems for HDR images.”  (Journal of Electronic Imaging, 1 September 2012)