Innehållsförteckning
List of Contributors XV
Foreword XIX
Part I Fundamentals 1
1 Introduction 3
Gabriel Cristóbal, Laurent U. Perrinet, and Matthias S. Keil
1.1 Why Should We Be Inspired by Biology? 4
1.2 Organization of Chapters in the Book 6
1.3 Conclusion 9
Acknowledgments 9
References 9
2 Bioinspired Vision Sensing 11
Christoph Posch
2.1 Introduction 11
2.1.1 Neuromorphic Engineering 12
2.1.2 Implementing Neuromorphic Systems 13
2.2 Fundamentals and Motivation: Bioinspired Artificial Vision 13
2.2.1 Limitations in Vision Engineering 14
2.2.2 The Human Retina from an Engineering Viewpoint 14
2.2.3 Modeling the Retina in Silicon 17
2.3 From Biological Models to Practical Vision Devices 18
2.3.1 The Wiring Problem 18
2.3.2 Where and What 20
2.3.3 Temporal Contrast: The DVS 21
2.3.4 Event-Driven Time-Domain Imaging: The ATIS 22
2.4 Conclusions and Outlook 25
References 26
3 Retinal Processing: From Biology to Models and Applications 29
David Alleysson and Nathalie Guyader
3.1 Introduction 29
3.2 Anatomy and Physiology of the Retina 30
3.2.1 Overview of the Retina 30
3.2.2 Photoreceptors 31
3.2.3 Outer and Inner Plexiform Layers (OPL and IPL) 33
3.2.4 Summary 34
3.3 Models of Vision 34
3.3.1 Overview of the Retina Models 34
3.3.2 Biological Models 35
3.3.2.1 Cellular and Molecular Models 35
3.3.2.2 Network Models 36
3.3.2.3 Parallel and Descriptive Models 38
3.3.3 Information Models 39
3.3.4 Geometry Models 40
3.4 Application to Digital Photography 42
3.4.1 Color Demosaicing 43
3.4.2 Color Constancy – Chromatic Adaptation – White Balance – Tone Mapping 44
3.5 Conclusion 45
References 46
4 Modeling Natural Image Statistics 53
Holly E. Gerhard, Lucas Theis, and Matthias Bethge
4.1 Introduction 53
4.2 Why Model Natural Images? 53
4.3 Natural Image Models 55
4.3.1 Model Evaluation 65
4.4 Computer Vision Applications 69
4.5 Biological Adaptations to Natural Images 71
4.6 Conclusions 75
References 76
5 Perceptual Psychophysics 81
C. Alejandro Parraga
5.1 Introduction 81
5.1.1 What Is Psychophysics and Why Do We Need It? 81
5.2 Laboratory Methods 82
5.2.1 Accuracy and Precision 83
5.2.2 Error Propagation 84
5.3 Psychophysical Threshold Measurement 85
5.3.1 Weber’s Law 85
5.3.2 Sensitivity Functions 86
5.4 Classic Psychophysics:Theory and Methods 86
5.4.1 Theory 87
5.4.2 Method of Constant Stimuli 88
5.4.3 Method of Limits 90
5.4.3.1 Forced-Choice Methods 92
5.4.4 Method of Adjustments 93
5.4.5 Estimating Psychometric Function Parameters 94
5.5 Signal Detection Theory 94
5.5.1 Signal and Noise 94
5.5.2 The Receiver Operating Characteristic 96
5.6 Psychophysical Scaling Methods 98
5.6.1 Discrimination Scales 99
5.6.2 Rating Scales 100
5.6.2.1 Equipartition Scales 100
5.6.2.2 Paired Comparison Scales 101
5.7 Conclusions 105
References 106
Part II Sensing 109
6 Bioinspired Optical Imaging 111
Mukul Sarkar
6.1 Visual Perception 111
6.1.1 Natural Single-Aperture and Multiple-Aperture Eyes 112
6.1.1.1 Human Eyes 113
6.1.1.2 Compound Eyes 114
6.1.1.3 Resolution 114
6.1.1.4 Visual Acuity 115
6.1.1.5 Depth Perception 115
6.1.1.6 Color Vision 116
6.1.1.7 Speed of Imaging and Motion Detection 117
6.1.1.8 Polarization Vision 117
6.2 Polarization Vision – Object Differentiation/Recognition 119
6.2.1 Polarization of Light 121
6.2.2 Polarization Imaging 125
6.2.2.1 Detection of Transparent and Opaque Objects 126
6.2.2.2 Shape Detection Using Polarized Light 131
6.3 High-Speed Motion Detection 133
6.3.1 Temporal Differentiation 135
6.4 Conclusion 138
References 139
7 Biomimetic Vision Systems 143
Reinhard Voelkel
7.1 Introduction 143
7.2 Scaling Laws in Optics 144
7.2.1 Optical Properties of Imaging Lens Systems 144
7.2.2 Space Bandwidth Product (SW) 146
7.3 The Evolution of Vision Systems 147
7.3.1 Single-Aperture Eyes 148
7.3.2 Compound Eyes 149
7.3.3 The Array Optics Concept 150
7.3.4 Jumping Spiders: Perfect Eyes for Small Animals 151
7.3.5 Nocturnal Spiders: Night Vision 153
7.4 Manufacturing of Optics for Miniaturized Vision Systems 154
7.4.1 Optics Industry 154
7.4.2 Planar Array Optics for Stereoscopic Vision 155
7.4.3 The Lack of a Suitable Fabrication Technology Hinders Innovation 155
7.4.4 Semiconductor Industry Promotes Wafer-Based Manufacturing 156
7.4.5 Image Sensors 156
7.4.6 Wafer-Based Manufacturing of Optics 158
7.4.7 Manufacturing of Microlens Arrays on Wafer Level 159
7.4.8 Diffractive Optical Elements and Subwavelength Structures for Antireflection Coatings 160
7.4.9 Microlens Imprint and Replication Processes 162
7.4.10 Wafer-Level Stacking or Packaging (WLP) 162
7.4.11 Wafer-Level Camera (WLC) 163
7.5 Examples for Biomimetic Compound Vision Systems 164
7.5.1 Ultraflat Cameras 165
7.5.2 Biologically Inspired Vision Systems for Smartphone Cameras 166
7.5.3 Pi Cam Cluster Camera 167
7.5.4 Panoramic Motion Camera for Flying Robots 169
7.5.5 Conclusion 170
References 172
8 Plenoptic Cameras 175
Fernando Pérez Nava, Alejandro Pérez Nava, Manuel Rodríguez Valido, and Eduardo Magdaleno Castellò
8.1 Introduction 175
8.2 Light Field Representation of the Plenoptic Function 177
8.2.1 The Plenoptic Function 177
8.2.2 Light Field Parameterizations 178
8.2.3 Light Field Reparameterization 179
8.2.4 Image Formation 180
8.2.5 Light Field Sampling 180
8.2.5.1 Pinhole and Thin Lens Camera 180
8.2.5.2 Multiple Devices 181
8.2.5.3 Temporal Multiplexing 181
8.2.5.4 Frequency Multiplexing 182
8.2.5.5 Spatial Multiplexing 182
8.2.5.6 Simulation 182
8.2.6 Light Field Sampling Analysis 183
8.2.7 Light Field Visualization 183
8.3 The Plenoptic Camera 185
8.4 Applications of the Plenoptic Camera 188
8.4.1 Refocusing 188
8.4.2 Perspective Shift 190
8.4.3 Depth Estimation 190
8.4.4 Extended Depth of Field Images 192
8.4.5 Superresolution 192
8.5 Generalizations of the Plenoptic Camera 193
8.6 High-Performance Computing with Plenoptic Cameras 195
8.7 Conclusions 196
References 197
Part III Modelling 201
9 Probabilistic Inference and Bayesian Priors in Visual Perception 203
Grigorios Sotiropoulos and Peggy Seriès
9.1 Introduction 203
9.2 Perception as Bayesian Inference 204
9.2.1 Deciding on a Single Percept 205
9.3 Perceptual Priors 207
9.3.1 Types of Prior Expectations 207
9.3.2 Impact of Expectations 208
9.3.3 The Slow Speed Prior 210
9.3.4 Expectations and Environmental Statistics 212
9.4 Outstanding Questions 214
9.4.1 Are Long-Term Priors Plastic? 214
9.4.2 How Specific are Priors? 215
9.4.3 Inference in Biological and Computer Vision 215
9.4.4 Conclusions 217
References 219
10 From Neuronal Models to Neuronal Dynamics and Image Processing 221
Matthias S. Keil
10.1 Introduction 221
10.2 The Membrane Equation as a Neuron Model 222
10.2.1 Synaptic Inputs 224
10.2.2 Firing Spikes 227
10.3 Application 1: A Dynamical Retinal Model 230
10.4 Application 2: Texture Segregation 234
10.5 Application 3: Detection of Collision Threats 236
10.6 Conclusions 239
Acknowledgments 240
References 240
11 Computational Models of Visual Attention and Applications 245
Olivier Le Meur and Matei Mancas
11.1 Introduction 245
11.2 Models of Visual Attention 246
11.2.1 Taxonomy 246
11.3 A Closer Look at Cognitive Models 250
11.3.1 Itti et al.’s Model [3] 250
11.3.2 Le Meur et al.’s Model [8] 251
11.3.2.1 Motivations 251
11.3.2.2 Global Architecture 252
11.3.3 Limitations 255
11.4 Applications 256
11.4.1 Saliency-Based Applications: A Brief Review 256
11.4.2 Predicting Memorability of Pictures 257
11.4.2.1 Memorability Definition 257
11.4.2.2 Memorability and Eye Movement 257
11.4.2.3 Computational Models 259
11.4.3 Quality Metric 260
11.4.3.1 Introduction 260
11.4.3.2 Eye Movement During a Quality Task 260
11.4.3.3 Saliency-Based Quality Metrics 261
11.5 Conclusion 262
References 262
12 Visual Motion Processing and Human Tracking Behavior 267
Anna Montagnini, Laurent U. Perrinet, and Guillaume S. Masson
12.1 Introduction 267
12.2 Pursuit Initiation: Facing Uncertainties 269
12.2.1 Where Is the Noise? Motion-Tracking Precision and Accuracy 269
12.2.2 Where Is the Target Really Going? 270
12.2.3 Human Smooth Pursuit as Dynamic Readout of the Neural Solution to the Aperture Problem 272
12.3 Predicting Future and On-Going Target Motion 273
12.3.1 Anticipatory Smooth Tracking 273
12.3.2 If You Don’t See It, You Can Still Predict (and Track) It 274
12.4 Dynamic Integration of Retinal and Extra-Retinal Motion Information: Computational Models 276
12.4.1 A Bayesian Approach for Open-Loop Motion Tracking 276
12.4.2 Bayesian (or Kalman-Filtering) Approach for Smooth Pursuit: Hierarchical Models 278
12.4.3 A Bayesian Approach for Smooth Pursuit: Dealing with Delays 279
12.5 Reacting, Inferring, Predicting: A Neural Workspace 282
12.6 Conclusion 286
12.6.1 Interest for Computer Vision 287
Acknowledgments 288
References 288
13 Cortical Networks of Visual Recognition 295
Christian Thériault, Nicolas Thome, and Matthieu Cord
13.1 Introduction 295
13.2 Global Organization of the Visual Cortex 296
13.3 Local Operations: Receptive Fields 297
13.4 Local Operations in V1 298
13.5 Multilayer Models 301
13.6 A Basic Introductory Model 302
13.7 Idealized Mathematical Model of V1: Fiber Bundle 307
13.8 Horizontal Connections and the Association Field 311
13.9 Feedback and Attentional Mechanisms 312
13.10 Temporal Considerations, Transformations and Invariance 312
13.11 Conclusion 314
References 315
14 Sparse Models for Computer Vision 319
Laurent U. Perrinet
14.1 Motivation 319
14.1.1 Efficiency and Sparseness in Biological Representations of Natural Images 319
14.1.2 Sparseness Induces Neural Organization 320
14.1.3 Outline: Sparse Models for Computer Vision 322
14.2 What Is Sparseness? Application to Image Patches 323
14.2.1 Definitions of Sparseness 323
14.2.2 Learning to Be Sparse:The Sparse Net Algorithm 325
14.2.3 Results: Efficiency of Different Learning Strategies 326
14.3 Sparse Lets: A Multiscale, Sparse, Biologically Inspired Representation of Natural Images 328
14.3.1 Motivation: Architecture of the Primary Visual Cortex 328
14.3.2 The Sparse Lets Framework 330
14.3.3 Efficiency of the Sparse Lets Framework 333
14.4 Sparse Edges: Introducing Prior Information 336
14.4.1 Using the Prior in First-Order Statistics of Edges 336
14.4.2 Using the Prior Statistics of Edge Co-Occurrences 338
14.5 Conclusion 341
Acknowledgments 341
References 342
15 Biologically Inspired Keypoints 347
Alexandre Alahi, Georges Goetz, and Emmanuel D’Angelo
15.1 Introduction 347
15.2 Definitions 349
15.3 What Does the Frond-End of the Visual System Tell Us? 350
15.3.1 The Retina 350
15.3.2 From Photoreceptors to Pixels 350
15.3.3 Visual Compression 351
15.3.4 Retinal Sampling Pattern 351
15.3.5 Scale-Space Representation 352
15.3.6 Difference of Gaussians as a Model for RGC-Receptive Fields 353
15.3.7 A Linear Nonlinear Model 354
15.3.8 Gabor-Like Filters 354
15.4 Bioplausible Keypoint Extraction 355
15.4.1 Scale-Invariant Feature Transform 355
15.4.2 Speeded-Up Robust Features 356
15.4.3 Center Surround Extrema 356
15.4.4 Features from Accelerated Segment Test 357
15.5 Biologically Inspired Keypoint Representation 357
15.5.1 Motivations 357
15.5.2 Dense Gabor-Like Descriptors 358
15.5.2.1 Scale-Invariant Feature Transform Descriptor 358
15.5.2.2 Speeded-Up Robust Features Descriptor 359
15.5.3 Sparse Gaussian Kernels 360
15.5.3.1 Local Binary Descriptors 360
15.5.3.2 Fast Retina Keypoint Descriptor 360
15.5.3.3 Fast Retina Keypoint Saccadic Matching 362
15.5.4 Fast Retina Keypoint versus Other Local Binary Descriptors 363
15.6 Qualitative Analysis: Visualizing Keypoint Information 363
15.6.1 Motivations 363
15.6.2 Binary Feature Reconstruction: From Bits to Image Patches 364
15.6.2.1 Feature Inversion as an Inverse Problem 364
15.6.2.2 Interest of the Retinal Descriptor 367
15.6.3 From Feature Visualization to Crowd-Sourced Object Recognition 368
15.7 Conclusions 370
References 371
Part IV Applications 375
16 Nightvision Based on a Biological Model 377
Magnus Oskarsson, Henrik Malm, and Eric Warrant
16.1 Introduction 377
16.1.1 Related Work 378
16.2 Why Is Vision Difficult in Dim Light? 380
16.3 Why Is Digital Imaging Difficult in Dim Light? 382
16.4 Solving the Problem of Imaging in Dim Light 383
16.4.1 Enhancing the Image 385
16.4.1.1 Visual Image Enhancement in the Retina 385
16.4.1.2 Digital Image Enhancement 387
16.4.2 Filtering the Image 388
16.4.2.1 Spatial and Temporal Summation in Higher Visual Processing 388
16.4.2.2 Structure Tensor Filtering of Digital Images 391
16.5 Implementation and Evaluation of the Night-Vision Algorithm 393
16.5.1 Adaptation of Parameters to Noise Levels 394
16.5.2 Parallelization and Computational Aspects 395
16.5.3 Considerations for Color 396
16.5.4 Experimental Results 397
16.6 Conclusions 399
Acknowledgment 400
References 401
17 Bioinspired Motion Detection Based on an FPGA Platform 405
Tim Köhler
17.1 Introduction 405
17.2 A Motion Detection Module for Robotics and Biology 406
17.3 Insect Motion Detection Models 407
17.4 Overview of Robotic Implementations of Bioinspired Motion Detection 412
17.4.1 Field-Programmable Gate Arrays (FPGA) 413
17.5 An FPGA-Based Implementation 414
17.5.1 FPGA-Camera-Module 414
17.5.2 A Configurable Array of EMDs 416
17.6 Experimental Results 419
17.7 Discussion 421
17.8 Conclusion 422
Acknowledgments 423
References 423
18 Visual Navigation in a Cluttered World 425
N. Andrew Browning and Florian Raudies
18.1 Introduction 425
18.2 Cues from Optic Flow: Visually Guided Navigation 426
18.3 Estimation of Self-Motion: Knowing Where You Are Going 429
18.4 Object Detection: Understanding What Is in Your Way 434
18.5 Estimation of TTC: Time Constraints from the Expansion Rate 439
18.6 Steering Control: The Importance of Representation 442
18.7 Conclusions 444
Acknowledgments 445
References 445
Index 447
