Imaging Life

Lawrence R. Griffing, PhD, is Associate Professor of Biology at Texas A&M University. He formerly served as Program Director for Cell Biology at the National Science Foundation and was Associate Director of the ITS-Center for Teaching and Learning. He teaches several undergraduate courses and graduate seminars on plant cell biology, 3D biology, and biological imaging.

• Preface xiiAcknowledgments xivAbout the Companion Website xvSection 1 Image Acquisition 11 Image Structure and Pixels 31.1 The Pixel Is the Smallest Discrete Unit of a Picture 31.2 The Resolving Power of a Camera or Display Is the Spatial Frequency of Its Pixels 61.3 Image Legibility Is the Ability to Recognize Text in an Image by Eye 71.4 Magnification Reduces Spatial Frequencies While Making Bigger Images 91.5 Technology Determines Scale and Resolution 111.6 The Nyquist Criterion: Capture at Twice the Spatial Frequency of the Smallest Object Imaged 121.7 Archival Time, Storage Limits, and the Resolution of the Display Medium Influence Capture and Scan Resolving Power 131.8 Digital Image Resizing or Scaling Match the Captured Image Resolution to the Output Resolution 141.9 Metadata Describes Image Content, Structure, and Conditions of Acquisition 162 Pixel Values and Image Contrast 202.1 Contrast Compares the Intensity of a Pixel with That of Its Surround 202.2 Pixel Values Determine Brightness and Color 212.3 The Histogram Is a Plot of the Number of Pixels in an Image at Each Level of Intensity 242.4 Tonal Range Is How Much of the Pixel Depth Is Used in an Image 252.5 The Image Histogram Shows Overexposure and Underexposure 262.6 High-Key Images Are Very Light, and Low-Key Images Are Very Dark 272.7 Color Images Have Various Pixel Depths 272.8 Contrast Analysis and Adjustment Using Histograms Are Available in Proprietary and Open-Source Software 292.9 The Intensity Transfer Graph Shows Adjustments of Contrast and Brightness Using Input and Output Histograms 302.10 Histogram Stretching Can Improve the Contrast and Tonal Range of the Image without Losing Information 322.11 Histogram Stretching of Color Channels Improves Color Balance 322.12 Software Tools for Contrast Manipulation Provide Linear, Non-linear, and Output-Visualized Adjustment 342.13 Different Image Formats Support Different Image Modes 362.14 Lossless Compression Preserves Pixel Values, and Lossy Compression Changes Them 373 Representation and Evaluation of Image Data 423.1 Image Representation Incorporates Multiple Visual Elements to Tell a Story 423.2 Illustrated Confections Combine the Accuracy of a Typical Specimen with a Science Story 423.3 Digital Confections Combine the Accuracy of Photography with a Science Story 453.4 The Video Storyboard Is an Explicit Visual Confection 483.5 Artificial Intelligence Can Generate Photorealistic Images from Text Stories 483.6 Making Images Believable: Show Representative Images and State the Acquisition Method 503.7 Making Images Understood: Clearly Identify Regions of Interest with Suitable Framing, Labels, and Image Contrast 513.8 Avoid Dequantification and Technical Artifacts While Not Hesitating to Take the Picture 553.9 Accurate, Reproducible Imaging Requires a Set of Rules and Guidelines 563.10 The Structural Similarity Index Measure Quantifies Image Degradation 574 Image Capture by Eye 614.1 The Anatomy of the Eye Limits Its Spatial Resolution 614.2 The Dynamic Range of the Eye Exceeds 11 Orders of Magnitude of Light Intensity, and Intrascene Dynamic Range Is about 3 Orders 634.3 The Absorption Characteristics of Photopigments of the Eye Determines Its Wavelength Sensitivity 634.4 Refraction and Reflection Determine the Optical Properties of Materials 674.5 Movement of Light Through the Eye Depends on the Refractive Index and Thickness of the Lens, the Vitreous Humor, and Other Components 694.6 Neural Feedback in the Brain Dictates Temporal Resolution of the Eye 694.7 We Sense Size and Distribution in Large Spaces Using the Rules of Perspective 704.8 Three-Dimensional Representation Depends on Eye Focus from Different Angles 714.9 Binocular Vision Relaxes the Eye and Provides a Three-Dimensional View in Stereomicroscopes 745 Image Capture with Digital Cameras 785.1 Digital Cameras are Everywhere 785.2 Light Interacts with Silicon Chips to Produce Electrons 785.3 The Anatomy of the Camera Chip Limits Its Spatial Resolution 805.4 Camera Chips Convert Spatial Frequencies to Temporal Frequencies with a Series of Horizontal and Vertical Clocks 825.5 Different Charge-Coupled Device Architectures Have Different Read-out Mechanisms 855.6 The Digital Camera Image Starts Out as an Analog Signal that Becomes Digital 875.7 Video Broadcast Uses Legacy Frequency Standards 885.8 Codecs Code and Decode Digital Video 895.9 Digital Video Playback Formats Vary Widely, Reflecting Different Means of Transmission and Display 915.10 The Light Absorption Characteristics of the Metal Oxide Semiconductor, Its Filters, and Its Coatings Determine the Wavelength Sensitivity of the Camera Chip 915.11 Camera Noise and Potential Well Size Determine the Sensitivity of the Camera to Detectable Light 935.12 Scientific Camera Chips Increase Light Sensitivity and Amplify the Signal 975.13 Cameras for Electron Microscopy Use Regular Imaging Chips after Converting Electrons to Photons or Detect the Electron Signal Directly with Modified CMOS 995.14 Camera Lenses Place Additional Constraints on Spatial Resolution 1015.15 Lens Aperture Controls Resolution, the Amount of Light, the Contrast, and the Depth of Field in a Digital Camera 1065.16 Relative Magnification with a Photographic Lens Depends on Chip Size and Lens Focal Length 1076 Image Capture by Scanning Systems 1116.1 Scanners Build Images Point by Point, Line by Line, and Slice by Slice 1116.2 Consumer-Grade Flatbed Scanners Provide Calibrated Color and Relatively High Resolution Over a Wide Field of View 1116.3 Scientific-Grade Flatbed Scanners Can Detect Chemiluminescence, Fluorescence, and Phosphorescence 1146.4 Scientific-Grade Scanning Systems Often Use Photomultiplier Tubes and Avalanche Photodiodes as the Camera 1186.5 X-ray Planar Radiography Uses Both Scanning and Camera Technologies 1196.6 Medical Computed Tomography Scans Rotate the X-ray Source and Sensor in a Helical Fashion Around the Body 1216.7 Micro-CT and Nano-CT Scanners Use Both Hard and Soft X-Rays and Can Resolve Cellular Features 1236.8 Macro Laser Scanners Acquire Three-Dimensional Images by Time-of-Flight or Structured Light 1256.9 Laser Scanning and Spinning Disks Generate Images for Confocal Scanning Microscopy 1266.10 Electron Beam Scanning Generates Images for Scanning Electron Microscopy 1286.11 Atomic Force Microscopy Scans a Force-Sensing Probe Across the Sample 128Section 2 Image Analysis 1357 Measuring Selected Image Features 1377.1 Digital Image Processing and Measurements are Part of the Image Metadata 1377.2 The Subject Matter Determines the Choice of Image Analysis and Measurement Software 1407.3 Recorded Paths, Regions of Interest, or Masks Save Selections for Measurement in Separate Images, Channels, and Overlays 1407.4 Stereology and Photoquadrat Sampling Measure Unsegmented Images 1447.5 Automatic Segmentation of Images Selects Image Features for Measurement Based on Common Feature Properties 1467.6 Segmenting by Pixel Intensity Is Thresholding 1467.7 Color Segmentation Looks for Similarities in a Three-Dimensional Color Space 1477.8 Morphological Image Processing Separates or Connects Features 1497.9 Measures of Pixel Intensity Quantify Light Absorption by and Emission from the Sample 1537.10 Morphometric Measurements Quantify the Geometric Properties of Selections 1557.11 Multi-dimensional Measurements Require Specific Filters 1568 Optics and Image Formation 1618.1 Optical Mechanics Can Be Well Described Mathematically 1618.2 A Lens Divides Space Into Image and Object Spaces 1618.3 The Lens Aperture Determines How Well the Lens Collects Radiation 1638.4 The Diffraction Limit and the Contrast between Two Closely Spaced Self-Luminous Spots Give Rise to the Limits of Resolution 1648.5 The Depth of the Three-Dimensional Slice of Object Space Remaining in Focus Is the Depth of Field 1678.6 In Electromagnetic Lenses, Focal Length Produces Focus and Magnification 1708.7 The Axial, Z-Dimensional, Point Spread Function Is a Measure of the Axial Resolution of High Numerical Aperture Lenses 1718.8 Numerical Aperture and Magnification Determine the Light-Gathering Properties of the Microscope Objective 1728.9 The Modulation (Contrast) Transfer Function Relates the Relative Contrast to Resolving Power in Fourier, or Frequency, Space 1728.10 The Point Spread Function Convolves the Object to Generate the Image 1768.11 Problems with the Focus of the Lens Arise from Lens Aberrations 1778.12 Refractive Index Mismatch in the Sample Produces Spherical Aberration 1828.13 Adaptive Optics Compensate for Refractive Index Changes and Aberration Introduced by Thick Samples 1839 Contrast and Tone Control 1899.1 The Subject Determines the Lighting 1899.2 Light Measurements Use Two Different Standards: Photometric and Radiometric Units 1909.3 The Light Emission and Contrast of Small Objects Limits Their Visibility 1949.4 Use the Image Histogram to Adjust the Trade-off Between Depth of Field and Motion Blur 1949.5 Use the Camera’s Light Meter to Detect Intrascene Dynamic Range and Set Exposure Compensation 1969.6 Light Sources Produce a Variety of Colors and Intensities That Determine the Quality of the Illumination 1979.7 Lasers and LEDs Provide Lighting with Specific Color and High Intensity 1999.8 Change Light Values with Absorption, Reflectance, Interference, and Polarizing Filters 2009.9 Köhler-Illuminated Microscopes Produce Conjugate Planes of Collimated Light from the Source and Specimen 2039.10 Reflectors, Diffusers, and Filters Control Lighting in Macro-imaging 20710 Processing with Digital Filters 21210.1 Image Processing Occurs Before, During, and After Image Acquisition 21210.2 Near-Neighbor Operations Modify the Value of a Target Pixel 21410.3 Rank Filters Identify Noise and Remove It from Images 21510.4 Convolution Can Be an Arithmetic Operation with Near Neighbors 21710.5 Deblurring and Background Subtraction Remove Out-of-Focus Features from Optical Sections 22110.6 Convolution Operations in Frequency Space Multiply the Fourier Transform of an Image by the Fourier Transform of the Convolution Mask 22210.7 Tomographic Operations in Frequency Space Produce Better Back-Projections 22410.8 Deconvolution in Frequency Space Removes Blur Introduced by the Optical System But Has a Problem with Noise 22411 Spatial Analysis 23111.1 Affine Transforms Produce Geometric Transformations 23111.2 Measuring Geometric Distortion Requires Grid Calibration 23111.3 Distortion Compensation Locally Adds and Subtracts Pixels 23111.4 Shape Analysis Starts with the Identification of Landmarks, Then Registration 23211.5 Grid Transformations are the Basis for Morphometric Examination of Shape Change in Populations 23411.6 Principal Component Analysis and Canonical Variates Analysis Use Measures of Similarity as Coordinates 23711.7 Convolutional Neural Networks Can Identify Shapes and Objects Using Deep Learning 23811.8 Boundary Morphometrics Analyzes and Mathematically Describes the Edge of the Object 24011.9 Measurement of Object Boundaries Can Reveal Fractal Relationships 24511.10 Pixel Intensity–Based Colocalization Analysis Reports the Spatial Correlation of Overlapping Signals 24611.11 Distance-Based Colocalization and Cluster Analysis Analyze the Spatial Proximity of Objects 25011.12 Fluorescence Resonance Energy Transfer Occurs Over Small (1–10 nm) Distances 25211.13 Image Correlations Reveal Patterns in Time and Space 25312 Temporal Analysis 26012.1 Representations of Molecular, Cellular, Tissue, and Organism Dynamics Require Video and Motion Graphics 26012.2 Motion Graphics Editors Use Key Frames to Specify Motion 26212.3 Motion Estimation Uses Successive Video Frames to Analyze Motion 26512.4 Optic Flow Compares the Intensities of Pixels, Pixel Blocks, or Regions Between Frames 26612.5 The Kymograph Uses Time as an Axis to Make a Visual Plot of the Object Motion 26812.6 Particle Tracking Is a Form of Feature-Based Motion Estimation 26912.7 Fluorescence Recovery After Photobleaching Shows Compartment Connectivity and the Movement of Molecules 27312.8 Fluorescence Switching Also Shows Connectivity and Movement 27612.9 Fluorescence Correlation Spectroscopy and Raster Image Correlation Spectroscopy Can Distinguish between Diffusion and Advection 28012.10 Fluorescent Protein Timers Provide Tracking of Maturing Proteins as They Move through Compartments 28213 Three-Dimensional Imaging, Modeling, and Analysis 28713.1 Three-Dimensional Worlds Are Scalable and Require Both Camera and Actor Views 28713.2 Stacking Multiple Adjacent Slices Can Produce a Three-Dimensional Volume or Surface 29113.3 Structure-from-Motion Photogrammetry Reconstructs Three-Dimensional Surfaces Using Multiple Camera Views 29213.4 Reconstruction of Aligned Images in Fourier Space Produces Three-Dimensional Volumes or Surfaces 29513.5 Surface Rendering Produces Isosurface Polygon Meshes Generated from Contoured Intensities 29613.6 Texture Maps of Object Isosurfaces Are Images or Movies 29913.7 Ray Tracing Follows a Ray of Light Backward from the Eye or Camera to Its Source 30013.8 Ray Tracing Shows the Object Based on Internal Intensities or Nearness to the Camera 30013.9 Transfer Functions Discriminate Objects in Ray-Traced Three Dimensions 30113.10 Four Dimensions, a Time Series of Three-Dimensional Volumes, Can Use Either Ray-Traced or Isosurface Rendering 30313.11 Volumes Rendered with Splats and Texture Maps Provide Realistic Object-Ordered Reconstructions 30313.12 Analysis of Three-Dimensional Volumes Uses the Same Approaches as Two-Dimensional Area Analysis But Includes Voxel Adjacency and Connectivity 30513.13 Head-Mounted Displays and Holograms Achieve an Immersive Three-Dimensional Experience 307Section 3 Image Modalities 31314 Ultrasound Imaging 31514.1 Ultrasonography Is a Cheap, High-Resolution, Deep-Penetration, Non-invasive Imaging Modality 31514.2 Many Species Use Ultrasound and Infrasound for Communication and Detection 31514.3 Sound Is a Compression, or Pressure, Wave 31614.4 The Measurement of Audible Sound Intensity Is in Decibels 31714.5 A Piezoelectric Transducer Creates the Ultrasound Wave 31814.6 Different Tissues Have Different Acoustic Impedances 31914.7 Sonic Wave Scatter Generates Speckle 32114.8 Lateral Resolution Depends on Sound Frequency and the Size and Focal Length of the Transducer Elements 32214.9 Axial Resolution Depends on the Duration of the Ultrasound Pulse 32314.10 Scatter and Absorption by Tissues Attenuate the Ultrasound Beam 32414.11 Amplitude Mode, Motion Mode, Brightness Mode, and Coherent Planar Wave Mode Are the Standard Modes for Clinical Practice 32414.12 Doppler Scans of Moving Red Blood Cells Reveal Changes in Vascular Flows with Time and Provide the Basis for Functional Ultrasound Imaging 32714.13 Microbubbles and Gas Vesicles Provide Ultrasound Contrast and Have Therapeutic Potential 32915 Magnetic Resonance Imaging 33415.1 Magnetic Resonance Imaging, Like Ultrasound, Performs Non-invasive Analysis without Ionizing Radiation 33415.2 Magnetic Resonance Imaging Is an Image of the Hydrogen Nuclei in Fat and Water 33715.3 Magnetic Resonance Imaging Sets up a Net Magnetization in Each Voxel That Is in Dynamic Equilibrium with the Applied Field 33815.4 The Magnetic Field Imposed by Magnetic Resonance Imaging Makes Protons Spin Like Tops with the Same Tilt and Determines the Frequency of Precession 33815.5 Magnetic Resonance Imaging Disturbs the Net Magnetization Equilibrium and Then Follows the Relaxation Back to Equilibrium 33915.6 T2 Relaxation, or Spin–Spin Relaxation, Causes the Disappearance of Transverse (x-y Direction) Magnetization Through Dephasing 34215.7 T1 Relaxation, or Spin-Lattice Relaxation, Causes the Disappearance of Longitudinal (z-Direction) Magnetization Through Energy Loss 34215.8 Faraday Induction Produces the Magnetic Resonance Imaging Signal (in Volts) with Coils in the x-y Plane 34315.9 Magnetic Gradients and Selective Radiofrequency Frequencies Generate Slices in the x, y, and z Directions 34315.10 Acquiring a Gradient Echo Image Is a Highly Repetitive Process, Getting Information Independently in the x, y, and z Dimensions 34415.11 Fast Low-Angle Shot Gradient Echo Imaging Speeds Up Imaging for T1-Weighted Images 34615.12 The Spin-Echo Image Compensates for Magnetic Heterogeneities in the Tissue in T2-Weighted Images 34615.13 Three-Dimensional Imaging Sequences Produce Higher Axial Resolution 34715.14 Echo Planar Imaging Is a Fast Two-Dimensional Imaging Modality But Has Limited Resolving Power 34715.15 Magnetic Resonance Angiography Analyzes Blood Velocity 34715.16 Diffusion Tensor Imaging Visualizes and Compares Directional (Anisotropic) Diffusion Coefficients in a Tissue 34915.17 Functional Magnetic Resonance Imaging Provides a Map of Brain Activity 35015.18 Magnetic Resonance Imaging Contrast Agents Detect Small Lesions That Are Otherwise Difficult to Detect 35116 Microscopy with Transmitted and Refracted Light 35516.1 Brightfield Microscopy of Living Cells Uses Apertures and the Absorbance of Transmitted Light to Generate Contrast 35516.2 Staining Fixed or Frozen Tissue Can Localize Large Polymers, Such as Proteins, Carbohydrates, and Nucleic Acids, But Is Less Effective for Lipids, Diffusible Ions, and Small Metabolites 36116.3 Darkfield Microscopy Generates Contrast by Only Collecting the Refracted Light from the Specimen 36516.4 Rheinberg Microscopy Generates Contrast by Producing Color Differences between Refracted and Unrefracted Light 36816.5 Wave Interference from the Object and Its Surround Generates Contrast in Polarized Light, Differential Interference Contrast, and Phase Contrast Microscopies 36916.6 Phase Contrast Microscopy Generates Contrast by Changing the Phase Difference Between the Light Coming from the Object and Its Surround 36916.7 Polarized Light Reveals Order within a Specimen and Differences in Object Thickness 37416.8 The Phase Difference Between the Slow and Fast Axes of Ordered Specimens Generates Contrast in Polarized Light Microscopy 37616.9 Compensators Cancel Out or Add to the Retardation Introduced by the Sample, Making It Possible to Measure the Sample Retardation 37916.10 Differential Interference Contrast Microscopy Is a Form of Polarized Light Microscopy That Generates Contrast Through Differential Interference of Two Slightly Separated Beams of Light 38317 Microscopy Using Fluoresced and Reflected Light 39017.1 Fluorescence and Autofluorescence: Excitation of Molecules by Light Leads to Rapid Re-emission of Lower Energy Light 39017.2 Fluorescence Properties Vary Among Molecules and Depend on Their Environment 39117.3 Fluorescent Labels Include Fluorescent Proteins, Fluorescent Labeling Agents, and Vital and Non-vital Fluorescence Affinity Dyes 39417.4 Fluorescence Environment Sensors Include Single-Wavelength Ion Sensors, Ratio Imaging Ion Sensors, FRET Sensors, and FRET-FLIM Sensors 39917.5 Widefield Microscopy for Reflective or Fluorescent Samples Uses Epi-illumination 40217.6 Epi-polarization Microscopy Detects Reflective Ordered Inorganic or Organic Crystallites and Uses Nanogold and Gold Beads as Labels 40517.7 To Optimize the Signal from the Sample, Use Specialized and Adaptive Optics 40517.8 Confocal Microscopes Use Accurate, Mechanical Four-Dimensional Epi-illumination and Acquisition 40817.9 The Best Light Sources for Fluorescence Match Fluorophore Absorbance 41017.10 Filters, Mirrors, and Computational Approaches Optimize Signal While Limiting the Crosstalk Between Fluorophores 41117.11 The Confocal Microscope Has Higher Axial and Lateral Resolving Power Than the Widefield Epi-illuminated Microscope, Some Designs Reaching Superresolution 41517.12 Multiphoton Microscopy and Other Forms of Non-linear Optics Create Conditions for Near-Simultaneous Excitation of Fluorophores with Two or More Photons 41918 Extending the Resolving Power of the Light Microscope in Time and Space 42718.1 Superresolution Microscopy Extends the Resolving Power of the Light Microscope 42718.2 Fluorescence Lifetime Imaging Uses a Temporal Resolving Power that Extends to Gigahertz Frequencies (Nanosecond Resolution) 42818.3 Spatial Resolving Power Extends Past the Diffraction Limit of Light 42918.4 Light Sheet Fluorescence Microscopy Achieves Fast Acquisition Times and Low Photon Dose 43218.5 Lattice Light Sheets Increase Axial Resolving Power 43518.6 Total Internal Reflection Microscopy and Glancing Incident Microscopy Produce a Thin Sheet of Excitation Energy Near the Coverslip 43718.7 Structured Illumination Microscopy Improves Resolution with Harmonic Patterns That Reveal Higher Spatial Frequencies 44018.8 Stimulated Emission Depletion and Reversible Saturable Optical Linear Fluorescence Transitions Superresolution Approaches Use Reversibly Saturable Fluorescence to Reduce the Size of the Illumination Spot 44718.9 Single-Molecule Excitation Microscopies, Photo-Activated Localization Microscopy, and Stochastic Optical Reconstruction Microscopy Also Rely on Switchable Fluorophores 45218.10 MINFLUX Combines Single-Molecule Localization with Structured Illumination to Get Resolution below 10 nm 45519 Electron Microscopy 46119.1 Electron Microscopy Uses a Transmitted Primary Electron Beam (Transmission Electron Micrography) or Secondary and Backscattered Electrons (Scanning Electron Micrography) to Image the Sample 46119.2 Some Forms of Scanning Electron Micrography Use Unfixed Tissue at Low Vacuums (Relatively High Pressure) 46219.3 Both Transmission Electron Micrography and Scanning Electron Micrography Use Frozen or Fixed Tissues 46519.4 Critical Point Drying and Surface Coating with Metal Preserves Surface Structures and Enhances Contrast for Scanning Electron Micrography 46719.5 Glass and Diamond Knives Make Ultrathin Sections on Ultramicrotomes 46819.6 The Filament Type and the Condenser Lenses Control Illumination in Scanning Electron Micrography and Transmission Electron Micrography 47119.7 The Objective Lens Aperture Blocks Scattered Electrons, Producing Contrast in Transmission Electron Micrography 47419.8 High-Resolution Transmission Electron Micrography Uses Large (or No) Objective Apertures 47519.9 Conventional Transmission Electron Micrography Provides a Cellular Context for Visualizing Organelles and Specific Molecules 47919.10 Serial Section Transmitted Primary Electron Analysis Can Provide Three-Dimensional Cellular Structures 48219.11 Scanning Electron Micrography Volume Microscopy Produces Three-Dimensional Microscopy at Nanometer Scales and Includes In-Lens Detectors and In-Column Sectioning Devices 48319.12 Correlative Electron Microscopy Provides Ultrastructural Context for Fluorescence Studies 48819.13 Tomographic Reconstruction of Transmission Electron Micrography Images Produces Very Thin (10-nm) Virtual Sections for High-Resolution Three-Dimensional Reconstruction 49019.14 Cryo-Electron Microscopy Achieves Molecular Resolving Power (Resolution, 0.1–0.2 Nm) Using Single-Particle Analysis 492Index 497