Computer Vision and Imaging in Intelligent Transportation Systems

Robert P. Loce, Conduent Labs, USADr. Robert P. Loce is a Fellow of SPIE and a Senior Member of IEEE. His publications include a book on enhancement and restoration of digital documents, and 8 book chapters on digital halftoning and digital document processing, 28 refereed journal publications, and 53 conference proceedings. He is currently an associate editor for Journal of Electronic Imaging, where he recently guest-edited a special topic issue on the subject matter of the proposed book.  He also chairs a conference within the SPIE/IS&T Electronic Imaging symposium on the subject matter of the proposed book.  He has also been an associate editor for Real-Time Imaging, and IEEE Transactions on Image Processing.Raja Bala, Samsung Research America, USADr. Bala has authored over 100 publications, including several book chapters, and holds over 120 U.S. patents in the field of digital and color imaging. He has served as adjunct faculty member at the Rochester Institute of Technology, and has taught many short courses and guest lectures on a variety of topics in digital imaging. From 2008-12, he served as Vice President of Publications for the Society for Imaging Science and Technology, where he led the Editorial Board for the IS&T/Wiley Book Series. He has served as Associate Editor of the Journal of Imaging Science and Technology, and is a frequent reviewer for IEEE Transactions on Image Processing, Journal of Electronic Imaging, and Journal of Imaging Science and Technology. Dr. Bala is a Fellow of IS&T and Senior Member of IEEE.Mohan Trivedi, Jacobs School of Engineering, University of California, San Diego, USAProf. Mohan Trivedi is the Head of UCSD's Computer Vision and Robotics Research laboratory, overseeing projects such as a robotic, sensor-based traffic-incident monitoring and response system (sponsored by Caltrans). Prof. Trivedi is leading an interdisciplinary effort, as UCSD layer leader for intelligent transportation and telematics for the California Institute for Telecommunications and Information Technology [Cal-(IT)2]. Prof. Trivedi is a recipient of the Pioneer Award and the Meritorious Service Award from the IEEE Computer Society; and the Distinguished Alumnus Award from Utah State University. He is a Fellow of the International Society for Optical Engineering (SPIE). He is a founding member of the Executive Committee of the UC System-wide Digital Media Innovation Program (DiMI). He is also Editor-in-Chief of Machine Vision & Applications (Springer).

• List of Contributors xiiiPreface xviiAcknowledgments xxiAbout the Companion Website xxiii1 Introduction 1Raja Bala and Robert P. Loce1.1 Law Enforcement and Security 11.2 Efficiency 41.3 Driver Safety and Comfort 51.4 A Computer Vision Framework for Transportation Applications 71.4.1 Image and Video Capture 81.4.2 Data Preprocessing 81.4.3 Feature Extraction 91.4.4 Inference Engine 101.4.5 Data Presentation and Feedback 11Part I Imaging from the Roadway Infrastructure 152 Automated License Plate Recognition 17Aaron Burry and Vladimir Kozitsky2.1 Introduction 172.2 Core ALPR Technologies 182.2.1 License Plate Localization 192.2.2 Character Segmentation 242.2.3 Character Recognition 282.2.4 State Identification 383 Vehicle Classification 47Shashank Deshpande, Wiktor Muron and Yang Cai3.1 Introduction 473.2 Overview of the Algorithms 483.3 Existing AVC Methods 483.4 LiDAR Imaging-Based 493.4.1 LiDAR Sensors 493.4.2 Fusion of LiDAR and Vision Sensors 503.5 Thermal Imaging-Based 533.5.1 Thermal Signatures 533.5.2 Intensity Shape-Based 563.6 Shape- and Profile-Based 583.6.1 Silhouette Measurements 603.6.2 Edge-Based Classification 653.6.3 Histogram of Oriented Gradients 673.6.4 Haar Features 683.6.5 Principal Component Analysis 693.7 Intrinsic Proportion Model 723.8 3D Model-Based Classification 743.9 SIFT-Based Classification 743.10 Summary 754 Detection of Passenger Compartment Violations 81Orhan Bulan, Beilei Xu, Robert P. Loce and Peter Paul4.1 Introduction 814.2 Sensing within the Passenger Compartment 824.2.1 Seat Belt Usage Detection 824.2.2 Cell Phone Usage Detection 834.2.3 Occupancy Detection 834.3 Roadside Imaging 844.3.1 Image Acquisition Setup 844.3.2 Image Classification Methods 854.3.3 Detection-Based Methods 945 Detection of Moving Violations 101Wencheng Wu, Orhan Bulan, Edgar A. Bernal and Robert P. Loce5.1 Introduction 1015.2 Detection of Speed Violations 1015.2.1 Speed Estimation from Monocular Cameras 1025.2.2 Speed Estimation from Stereo Cameras 1085.2.3 Discussion 1155.3 Stop Violations 1155.3.1 Red Light Cameras 1155.4 Other Violations 1255.4.1 Wrong-Way Driver Detection 1255.4.2 Crossing Solid Lines 1266 Traffic Flow Analysis 131Rodrigo Fernandez, Muhammad Haroon Yousaf, Timothy J. Ellis, Zezhi Chen and Sergio A. Velastin6.1 What is Traffic Flow Analysis? 1316.1.1 Traffic Conflicts and Traffic Analysis 1316.1.2 Time Observation 1326.1.3 Space Observation 1336.1.4 The Fundamental Equation 1336.1.5 The Fundamental Diagram 1336.1.6 Measuring Traffic Variables 1346.1.7 Road Counts 1356.1.8 Junction Counts 1356.1.9 Passenger Counts 1366.1.10 Pedestrian Counts 1366.1.11 Speed Measurement 1366.2 The Use of Video Analysis in Intelligent Transportation Systems 1376.2.1 Introduction 1376.2.2 General Framework for Traffic Flow Analysis 1376.2.3 Application Domains 1436.3 Measuring Traffic Flow from Roadside CCTV Video 1446.3.1 Video Analysis Framework 1446.3.2 Vehicle Detection 1466.3.3 Background Model 1466.3.4 Counting Vehicles 1496.3.5 Tracking 1506.3.6 Camera Calibration 1506.3.7 Feature Extraction and Vehicle Classification 1526.3.8 Lane Detection 1536.3.9 Results 1556.4 Some Challenges 1567 Intersection Monitoring Using Computer Vision Techniques for Capacity, Delay, and Safety Analysis 163Brendan Tran Morris and Mohammad Shokrolah Shirazi7.1 Vision-Based Intersection Analysis: Capacity, Delay, and Safety 1637.1.1 Intersection Monitoring 1637.1.2 Computer Vision Application 1647.2 System Overview 1657.2.1 Tracking Road Users 1667.2.2 Camera Calibration 1697.3 Count Analysis 1717.3.1 Vehicular Counts 1717.3.2 Nonvehicular Counts 1737.4 Queue Length Estimation 1737.4.1 Detection-Based Methods 1747.4.2 Tracking-Based Methods 1757.5 Safety Analysis 1777.5.1 Behaviors 1787.5.2 Accidents 1827.5.3 Conflicts 1857.6 Challenging Problems and Perspectives 1877.6.1 Robust Detection and Tracking 1877.6.2 Validity of Prediction Models for Conflict and Collisions 1887.6.3 Cooperating Sensing Modalities 1897.6.4 Networked Traffic Monitoring Systems 1897.7 Conclusion 1898 Video-Based Parking Management 195Oliver Sidla and Yuriy Lipetski8.1 Introduction 1958.2 Overview of Parking Sensors 1978.3 Introduction to Vehicle Occupancy Detection Methods 2008.4 Monocular Vehicle Detection 2008.4.1 Advantages of Simple 2D Vehicle Detection 2008.4.2 Background Model–Based Approaches 2008.4.3 Vehicle Detection Using Local Feature Descriptors 2028.4.4 Appearance-Based Vehicle Detection 2038.4.5 Histograms of Oriented Gradients 2048.4.6 LBP Features and LBP Histograms 2078.4.7 Combining Detectors into Cascades and Complex Descriptors 2088.4.8 Case Study: Parking Space Monitoring Using a Combined Feature Detector 2088.4.9 Detection Using Artificial Neural Networks 2118.5 Introduction to Vehicle Detection with 3D Methods 2138.6 Stereo Vision Methods 2158.6.1 Introduction to Stereo Methods 2158.6.2 Limits on the Accuracy of Stereo Reconstruction 2168.6.3 Computing the Stereo Correspondence 2178.6.4 Simple Stereo for Volume Occupation Measurement 2188.6.5 A Practical System for Parking Space Monitoring Using a Stereo System 2188.6.6 Detection Methods Using Sparse 3D Reconstruction 2209 Video Anomaly Detection 227Raja Bala and Vishal Monga9.1 Introduction 2279.2 Event Encoding 2289.2.1 Trajectory Descriptors 2299.2.2 Spatiotemporal Descriptors 2319.3 Anomaly Detection Models 2339.3.1 Classification Methods 2339.3.2 Hidden Markov Models 2349.3.3 Contextual Methods 2349.4 Sparse Representation Methods for Robust Video Anomaly Detection 2369.4.1 Structured Anomaly Detection 2379.4.2 Unstructured Video Anomaly Detection 2439.4.3 Experimental Setup and Results 2459.5 Conclusion and Future Research 253Part II Imaging from and within the Vehicle 25710 Pedestrian Detection 259Shashank Deshpande and Yang Cai10.1 Introduction 25910.2 Overview of the Algorithms 25910.3 Thermal Imaging 26010.4 Background Subtraction Methods 26110.4.1 Frame Subtraction 26110.4.2 Approximate Median 26210.4.3 Gaussian Mixture Model 26310.5 Polar Coordinate Profile 26310.6 Image-Based Features 26510.6.1 Histogram of Oriented Gradients 26510.6.2 Deformable Parts Model 26610.6.3 LiDAR and Camera Fusion–Based Detection 26610.7 LiDAR Features 26810.7.1 Preprocessing Module 26810.7.2 Feature Extraction Module 26810.7.3 Fusion Module 26810.7.4 LIPD Dataset 27010.7.5 Overview of the Algorithm 27010.7.6 LiDAR Module 27210.7.7 Vision Module 27510.7.8 Results and Discussion 27610.7.8.1 LiDAR Module 27610.7.8.2 Vision Module 27610.8 Summary 28011 Lane Detection and Tracking Problems in Lane Departure Warning Systems 283Gianni Cario, Alessandro Casavola and Marco Lupia11.1 Introduction 28311.2 LD: Algorithms for a Single Frame 28511.2.1 Image Preprocessing 28511.2.2 Edge Extraction 28711.2.3 Stripe Identification 29111.2.4 Line Fitting 29411.3 LT Algorithms 29711.3.1 Recursive Filters on Subsequent N frames 29811.3.2 Kalman Filter 29811.4 Implementation of an LD and LT Algorithm 29911.4.1 Simulations 30011.4.2 Test Driving Scenario 30011.4.3 Driving Scenario: Lane Departures at Increasing Longitudinal Speed 30011.4.4 The Proposed Algorithm 30211.4.5 Conclusions 30312 Vision-Based Integrated Techniques for Collision Avoidance Systems 305Ravi Satzoda and Mohan Trivedi12.1 Introduction 30512.2 Related Work 30712.3 Context Definition for Integrated Approach 30712.4 ELVIS: Proposed Integrated Approach 30812.4.1 Vehicle Detection Using Lane Information 30912.4.2 Improving Lane Detection using On-Road Vehicle Information 31212.5 Performance Evaluation 31312.5.1 Vehicle Detection in ELVIS 31312.5.2 Lane Detection in ELVIS 31612.6 Concluding Remarks 31913 Driver Monitoring 321Raja Bala and Edgar A. Bernal13.1 Introduction 32113.2 Video Acquisition 32213.3 Face Detection and Alignment 32313.4 Eye Detection and Analysis 32513.5 Head Pose and Gaze Estimation 32613.5.1 Head Pose Estimation 32613.5.2 Gaze Estimation 32813.6 Facial Expression Analysis 33213.7 Multimodal Sensing and Fusion 33413.8 Conclusions and Future Directions 33614 Traffic Sign Detection and Recognition 343Hasan Fleyeh14.1 Introduction 34314.2 Traffic Signs 34414.2.1 The European Road and Traffic Signs 34414.2.2 The American Road and Traffic Signs 34714.3 Traffic Sign Recognition 34714.4 Traffic Sign Recognition Applications 34814.5 Potential Challenges 34914.6 Traffic Sign Recognition System Design 34914.6.1 Traffic Signs Datasets 35214.6.2 Colour Segmentation 35414.6.3 Traffic Sign's Rim Analysis 35914.6.4 Pictogram Extraction 36414.6.5 Pictogram Classification Using Features 36514.7 Working Systems 36915 Road Condition Monitoring 375Matti Kutila, Pasi Pyykonen, Johan Casselgren and Patrik Jonsson15.1 Introduction 37515.2 Measurement Principles 37615.3 Sensor Solutions 37715.3.1 Camera-Based Friction Estimation Systems 37715.3.2 Pavement Sensors 37915.3.3 Spectroscopy 38015.3.4 Roadside Fog Sensing 38215.3.5 In-Vehicle Sensors 38315.4 Classification and Sensor Fusion 38615.5 Field Studies 39015.6 Cooperative Road Weather Services 39415.7 Discussion and Future Work 395Index 399