Robert P. Loce & Raja Bala 
Computer Vision and Imaging in Intelligent Transportation Systems [EPUB ebook] 

List of Contributors xiii
Preface xvii
Acknowledgments xxi
About the Companion Website xxiii
1 Introduction 1
Raja Bala and Robert P. Loce
1.1 Law Enforcement and Security 1
1.2 Efficiency 4
1.3 Driver Safety and Comfort 5
1.4 A Computer Vision Framework for Transportation Applications 7
1.4.1 Image and Video Capture 8
1.4.2 Data Preprocessing 8
1.4.3 Feature Extraction 9
1.4.4 Inference Engine 10
1.4.5 Data Presentation and Feedback 11
Part I Imaging from the Roadway Infrastructure 15
2 Automated License Plate Recognition 17
Aaron Burry and Vladimir Kozitsky
2.1 Introduction 17
2.2 Core ALPR Technologies 18
2.2.1 License Plate Localization 19
2.2.2 Character Segmentation 24
2.2.3 Character Recognition 28
2.2.4 State Identification 38
3 Vehicle Classification 47
Shashank Deshpande, Wiktor Muron and Yang Cai
3.1 Introduction 47
3.2 Overview of the Algorithms 48
3.3 Existing AVC Methods 48
3.4 Li DAR Imaging-Based 49
3.4.1 Li DAR Sensors 49
3.4.2 Fusion of Li DAR and Vision Sensors 50
3.5 Thermal Imaging?-Based 53
3.5.1 Thermal Signatures 53
3.5.2 Intensity Shape?-Based 56
3.6 Shape?- and Profile?-Based 58
3.6.1 Silhouette Measurements 60
3.6.2 Edge?-Based Classification 65
3.6.3 Histogram of Oriented Gradients 67
3.6.4 Haar Features 68
3.6.5 Principal Component Analysis 69
3.7 Intrinsic Proportion Model 72
3.8 3D Model?-Based Classification 74
3.9 SIFT?-Based Classification 74
3.10 Summary 75
4 Detection of Passenger Compartment Violations 81
Orhan Bulan, Beilei Xu, Robert P. Loce and Peter Paul
4.1 Introduction 81
4.2 Sensing within the Passenger Compartment 82
4.2.1 Seat Belt Usage Detection 82
4.2.2 Cell Phone Usage Detection 83
4.2.3 Occupancy Detection 83
4.3 Roadside Imaging 84
4.3.1 Image Acquisition Setup 84
4.3.2 Image Classification Methods 85
4.3.3 Detection?-Based Methods 94
5 Detection of Moving Violations 101
Wencheng Wu, Orhan Bulan, Edgar A. Bernal and Robert P. Loce
5.1 Introduction 101
5.2 Detection of Speed Violations 101
5.2.1 Speed Estimation from Monocular Cameras 102
5.2.2 Speed Estimation from Stereo Cameras 108
5.2.3 Discussion 115
5.3 Stop Violations 115
5.3.1 Red Light Cameras 115
5.4 Other Violations 125
5.4.1 Wrong?-Way Driver Detection 125
5.4.2 Crossing Solid Lines 126
6 Traffic Flow Analysis 131
Rodrigo Fernandez, Muhammad Haroon Yousaf, Timothy J. Ellis, Zezhi Chen and Sergio A. Velastin
6.1 What is Traffic Flow Analysis? 131
6.1.1 Traffic Conflicts and Traffic Analysis 131
6.1.2 Time Observation 132
6.1.3 Space Observation 133
6.1.4 The Fundamental Equation 133
6.1.5 The Fundamental Diagram 133
6.1.6 Measuring Traffic Variables 134
6.1.7 Road Counts 135
6.1.8 Junction Counts 135
6.1.9 Passenger Counts 136
6.1.10 Pedestrian Counts 136
6.1.11 Speed Measurement 136
6.2 The Use of Video Analysis in Intelligent Transportation Systems 137
6.2.1 Introduction 137
6.2.2 General Framework for Traffic Flow Analysis 137
6.2.3 Application Domains 143
6.3 Measuring Traffic Flow from Roadside CCTV Video 144
6.3.1 Video Analysis Framework 144
6.3.2 Vehicle Detection 146
6.3.3 Background Model 146
6.3.4 Counting Vehicles 149
6.3.5 Tracking 150
6.3.6 Camera Calibration 150
6.3.7 Feature Extraction and Vehicle Classification 152
6.3.8 Lane Detection 153
6.3.9 Results 155
6.4 Some Challenges 156
7 Intersection Monitoring Using Computer Vision Techniques for Capacity, Delay, and Safety Analysis 163
Brendan Tran Morris and Mohammad Shokrolah Shirazi
7.1 Vision?-Based Intersection Analysis: Capacity, Delay, and Safety 163
7.1.1 Intersection Monitoring 163
7.1.2 Computer Vision Application 164
7.2 System Overview 165
7.2.1 Tracking Road Users 166
7.2.2 Camera Calibration 169
7.3 Count Analysis 171
7.3.1 Vehicular Counts 171
7.3.2 Nonvehicular Counts 173
7.4 Queue Length Estimation 173
7.4.1 Detection?-Based Methods 174
7.4.2 Tracking?-Based Methods 175
7.5 Safety Analysis 177
7.5.1 Behaviors 178
7.5.2 Accidents 182
7.5.3 Conflicts 185
7.6 Challenging Problems and Perspectives 187
7.6.1 Robust Detection and Tracking 187
7.6.2 Validity of Prediction Models for Conflict and Collisions 188
7.6.3 Cooperating Sensing Modalities 189
7.6.4 Networked Traffic Monitoring Systems 189
7.7 Conclusion 189
8 Video?-Based Parking Management 195
Oliver Sidla and Yuriy Lipetski
8.1 Introduction 195
8.2 Overview of Parking Sensors 197
8.3 Introduction to Vehicle Occupancy Detection Methods 200
8.4 Monocular Vehicle Detection 200
8.4.1 Advantages of Simple 2D Vehicle Detection 200
8.4.2 Background Model-Based Approaches 200
8.4.3 Vehicle Detection Using Local Feature Descriptors 202
8.4.4 Appearance?-Based Vehicle Detection 203
8.4.5 Histograms of Oriented Gradients 204
8.4.6 LBP Features and LBP Histograms 207
8.4.7 Combining Detectors into Cascades and Complex Descriptors 208
8.4.8 Case Study: Parking Space Monitoring Using a Combined Feature Detector 208
8.4.9 Detection Using Artificial Neural Networks 211
8.5 Introduction to Vehicle Detection with 3D Methods 213
8.6 Stereo Vision Methods 215
8.6.1 Introduction to Stereo Methods 215
8.6.2 Limits on the Accuracy of Stereo Reconstruction 216
8.6.3 Computing the Stereo Correspondence 217
8.6.4 Simple Stereo for Volume Occupation Measurement 218
8.6.5 A Practical System for Parking Space Monitoring Using a Stereo System 218
8.6.6 Detection Methods Using Sparse 3D Reconstruction 220
9 Video Anomaly Detection 227
Raja Bala and Vishal Monga
9.1 Introduction 227
9.2 Event Encoding 228
9.2.1 Trajectory Descriptors 229
9.2.2 Spatiotemporal Descriptors 231
9.3 Anomaly Detection Models 233
9.3.1 Classification Methods 233
9.3.2 Hidden Markov Models 234
9.3.3 Contextual Methods 234
9.4 Sparse Representation Methods for Robust Video Anomaly Detection 236
9.4.1 Structured Anomaly Detection 237
9.4.2 Unstructured Video Anomaly Detection 243
9.4.3 Experimental Setup and Results 245
9.5 Conclusion and Future Research 253
Part II Imaging from and within the Vehicle 257
10 Pedestrian Detection 259
Shashank Deshpande and Yang Cai
10.1 Introduction 259
10.2 Overview of the Algorithms 259
10.3 Thermal Imaging 260
10.4 Background Subtraction Methods 261
10.4.1 Frame Subtraction 261
10.4.2 Approximate Median 262
10.4.3 Gaussian Mixture Model 263
10.5 Polar Coordinate Profile 263
10.6 Image?-Based Features 265
10.6.1 Histogram of Oriented Gradients 265
10.6.2 Deformable Parts Model 266
10.6.3 Li DAR and Camera Fusion-Based Detection 266
10.7 Li DAR Features 268
10.7.1 Preprocessing Module 268
10.7.2 Feature Extraction Module 268
10.7.3 Fusion Module 268
10.7.4 LIPD Dataset 270
10.7.5 Overview of the Algorithm 270
10.7.6 Li DAR Module 272
10.7.7 Vision Module 275
10.7.8 Results and Discussion 276
10.7.8.1 Li DAR Module 276
10.7.8.2 Vision Module 276
10.8 Summary 280
11 Lane Detection and Tracking Problems in Lane Departure Warning Systems 283
Gianni Cario, Alessandro Casavola and Marco Lupia
11.1 Introduction 283
11.2 LD: Algorithms for a Single Frame 285
11.2.1 Image Preprocessing 285
11.2.2 Edge Extraction 287
11.2.3 Stripe Identification 291
11.2.4 Line Fitting 294
11.3 LT Algorithms 297
11.3.1 Recursive Filters on Subsequent N frames 298
11.3.2 Kalman Filter 298
11.4 Implementation of an LD and LT Algorithm 299
11.4.1 Simulations 300
11.4.2 Test Driving Scenario 300
11.4.3 Driving Scenario: Lane Departures at Increasing Longitudinal Speed 300
11.4.4 The Proposed Algorithm 302
11.4.5 Conclusions 303
12 Vision?-Based Integrated Techniques for Collision Avoidance Systems 305
Ravi Satzoda and Mohan Trivedi
12.1 Introduction 305
12.2 Related Work 307
12.3 Context Definition for Integrated Approach 307
12.4 ELVIS: Proposed Integrated Approach 308
12.4.1 Vehicle Detection Using Lane Information 309
12.4.2 Improving Lane Detection using On?-Road Vehicle Information 312
12.5 Performance Evaluation 313
12.5.1 Vehicle Detection in ELVIS 313
12.5.2 Lane Detection in ELVIS 316
12.6 Concluding Remarks 319
13 Driver Monitoring 321
Raja Bala and Edgar A. Bernal
13.1 Introduction 321
13.2 Video Acquisition 322
13.3 Face Detection and Alignment 323
13.4 Eye Detection and Analysis 325
13.5 Head Pose and Gaze Estimation 326
13.5.1 Head Pose Estimation 326
13.5.2 Gaze Estimation 328
13.6 Facial Expression Analysis 332
13.7 Multimodal Sensing and Fusion 334
13.8 Conclusions and Future Directions 336
14 Traffic Sign Detection and Recognition 343
Hasan Fleyeh
14.1 Introduction 343
14.2 Traffic Signs 344
14.2.1 The European Road and Traffic Signs 344
14.2.2 The American Road and Traffic Signs 347
14.3 Traffic Sign Recognition 347
14.4 Traffic Sign Recognition Applications 348
14.5 Potential Challenges 349
14.6 Traffic Sign Recognition System Design 349
14.6.1 Traffic Signs Datasets 352
14.6.2 Colour Segmentation 354
14.6.3 Traffic Sign’s Rim Analysis 359
14.6.4 Pictogram Extraction 364
14.6.5 Pictogram Classification Using Features 365
14.7 Working Systems 369
15 Road Condition Monitoring 375
Matti Kutila, Pasi Pyykonen, Johan Casselgren and Patrik Jonsson
15.1 Introduction 375
15.2 Measurement Principles 376
15.3 Sensor Solutions 377
15.3.1 Camera?-Based Friction Estimation Systems 377
15.3.2 Pavement Sensors 379
15.3.3 Spectroscopy 380
15.3.4 Roadside Fog Sensing 382
15.3.5 In?-Vehicle Sensors 383
15.4 Classification and Sensor Fusion 386
15.5 Field Studies 390
15.6 Cooperative Road Weather Services 394
15.7 Discussion and Future Work 395
Index 399

Robert P. Loce, Conduent Labs, USA
Dr. Robert P. Loce is a Fellow of SPIE and a Senior Member of IEEE. His publications include a book on enhancement and restoration of digital documents, and 8 book chapters on digital halftoning and digital document processing, 28 refereed journal publications, and 53 conference proceedings. He is currently an associate editor for Journal of Electronic Imaging, where he recently guest-edited a special topic issue on the subject matter of the proposed book. He also chairs a conference within the SPIE/IS&T Electronic Imaging symposium on the subject matter of the proposed book. He has also been an associate editor for Real-Time Imaging, and IEEE Transactions on Image Processing.
Raja Bala, Samsung Research America, USA
Dr. Bala has authored over 100 publications, including several book chapters, and holds over 120 U.S. patents in the field of digital and color imaging. He has served as adjunct faculty member at the Rochester Institute of Technology, and has taught many short courses and guest lectures on a variety of topics in digital imaging. From 2008-12, he served as Vice President of Publications for the Society for Imaging Science and Technology, where he led the Editorial Board for the IS&T/Wiley Book Series. He has served as Associate Editor of the Journal of Imaging Science and Technology, and is a frequent reviewer for IEEE Transactions on Image Processing, Journal of Electronic Imaging, and Journal of Imaging Science and Technology. Dr. Bala is a Fellow of IS&T and Senior Member of IEEE.
Mohan Trivedi, Jacobs School of Engineering, University of California, San Diego, USA
Prof. Mohan Trivedi is the Head of UCSD’s Computer Vision and Robotics Research laboratory, overseeing projects such as a robotic, sensor-based traffic-incident monitoring and response system (sponsored by Caltrans). Prof. Trivedi is leading an interdisciplinary effort, as UCSD layer leader for intelligent transportation and telematics for the California Institute for Telecommunications and Information Technology [Cal-(IT)2]. Prof. Trivedi is a recipient of the Pioneer Award and the Meritorious Service Award from the IEEE Computer Society; and the Distinguished Alumnus Award from Utah State University. He is a Fellow of the International Society for Optical Engineering (SPIE). He is a founding member of the Executive Committee of the UC System-wide Digital Media Innovation Program (Di MI). He is also Editor-in-Chief of Machine Vision & Applications (Springer).