"Computer Vision: A Modern Approach" By David A. Forsyth, Jean Ponce (Repost)

This book gives both a general view of the entire computer vision enterprise and also offers sufficient detail to be able to build useful applications. Users learn techniques that have proven to be useful by first-hand experience and a wide range of mathematical methods.

This book includes essential topics that either reflect practical significance or are of theoretical importance. Topics are discussed in substantial and increasing depth. Application surveys describe numerous important application areas such as image based rendering and digital libraries. Many important algorithms broken down and illustrated in pseudo code. Appropriate for use by engineers as a comprehensive reference to the computer vision enterprise.

Contents
I IMAGE FORMATION
1 RADIOMETRY - MEASURING LIGHT
1.1 Light in Space
1.1.1 Foreshortening
1.1.2 Solid Angle
1.1.3 Radiance
1.2 Light at Surfaces
1.2.1 Simplifying Assumptions
1.2.2 The Bidirectional Reflectance Distribution Function
1.3 Important Special Cases
1.3.1 Radiosity
1.3.2 Directional Hemispheric Reflectance
1.3.3 Lambertian Surfaces and Albedo
1.3.4 Specular Surfaces
1.3.5 The Lambertian + Specular Model
1.4 Quick Reference: Radiometric Terminology for Light
1.5 Quick Reference: Radiometric Properties of Surfaces
1.6 Quick Reference: Important Types of Surface
1.7 Notes
1.8 Assignments
2 SOURCES, SHADOWS AND SHADING
2.1 Radiometric Properties of Light Sources
2.2 Qualitative Radiometry
2.3 Sources and their Effects
2.3.1 Point Sources
2.3.2 Line Sources
2.3.3 Area Sources
2.4 Local Shading Models
2.4.1 Local Shading Models for Point Sources
2.4.2 Area Sources and their Shadows
2.4.3 Ambient Illumination
2.5 Application: Photometric Stereo
2.5.1 Normal and Albedo from Many Views
2.5.2 Shape from Normals
2.6 Interreflections: Global Shading Models
2.6.1 An Interreflection Model
2.6.2 Solving for Radiosity
2.6.3 The qualitative effects of interreflections
2.7 Notes
2.8 Assignments
2.8.1 Exercises
2.8.2 Programming Assignments
3 COLOUR
3.1 The Physics of Colour
3.1.1 Radiometry for Coloured Lights: Spectral Quantities
3.1.2 The Colour of Surfaces
3.1.3 The Colour of Sources
3.2 Human Colour Perception
3.2.1 Colour Matching
3.2.2 Colour Receptors
3.3 Representing Colour
3.3.1 Linear Colour Spaces
3.3.2 Non-linear Colour Spaces
3.3.3 Spatial and Temporal Effects
3.4 Application: Finding Specularities
3.5 Surface Colour from Image Colour
3.5.1 Surface Colour Perception in People
3.5.2 Inferring Lightness
3.5.3 A Model for Image Colour
3.5.4 Surface Colour from Finite Dimensional Linear Models
3.6 Notes
3.6.1 Trichromacy and Colour Spaces
3.6.2 Lightness and Colour Constancy
3.6.3 Colour in Recognition
3.7 Assignments
II IMAGE MODELS
4 GEOMETRIC IMAGE FEATURES
4.1 Elements of Differential Geometry
4.1.1 Curves
4.1.2 Surfaces
Application: The shape of specularities
4.2 Contour Geometry
4.2.1 The Occluding Contour and the Image Contour
4.2.2 The Cusps and Inflections of the Image Contour
4.2.3 Koenderink’s Theorem
4.3 Notes
4.4 Assignments
5 ANALYTICAL IMAGE FEATURES
5.1 Elements of Analytical Euclidean Geometry
5.1.1 Coordinate Systems and Homogeneous Coordinates
5.1.2 Coordinate System Changes and Rigid Transformations
5.2 Geometric Camera Parameters
5.2.1 Intrinsic Parameters
5.2.2 Extrinsic Parameters
5.2.3 A Characterization of Perspective Projection Matrices
5.3 Calibration Methods
5.3.1 A Linear Approach to Camera Calibration
Technique: Linear Least Squares Methods
5.3.2 Taking Radial Distortion into Account
5.3.3 Using Straight Lines for Calibration
5.3.4 Analytical Photogrammetry
Technique: Non-Linear Least Squares Methods
5.4 Notes
5.5 Assignments
6 AN INTRODUCTION TO PROBABILITY
6.1 Probability in Discrete Spaces
6.1.1 Probability: the P-function
6.1.2 Conditional Probability
6.1.3 Choosing P
6.2 Probability in Continuous Spaces
6.2.1 Event Structures for Continuous Spaces
6.2.2 Representing a P-function for the Real Line
6.2.3 Probability Densities
6.3 Random Variables
6.3.1 Conditional Probability and Independence
6.3.2 Expectations
6.3.3 Joint Distributions and Marginalization
6.4 Standard Distributions and Densities
6.4.1 The Normal Distribution
6.5 Probabilistic Inference
6.5.1 The Maximum Likelihood Principle
6.5.2 Priors, Posteriors and Bayes’ rule
6.5.3 Bayesian Inference
6.5.4 Open Issues
6.6 Discussion
III EARLY VISION: ONE IMAGE
7 LINEAR FILTERS
7.1 Linear Filters and Convolution
7.1.1 Convolution
7.1.2 Example: Smoothing by Averaging
7.1.3 Example: Smoothing with a Gaussian
7.2 Shift invariant linear systems
7.2.1 Discrete Convolution
7.2.2 Continuous Convolution
7.2.3 Edge Effects in Discrete Convolutions
7.3 Spatial Frequency and Fourier Transforms
7.3.1 Fourier Transforms
7.4 Sampling and Aliasing
7.4.1 Sampling
7.4.2 Aliasing
7.4.3 Smoothing and Resampling
7.5 Technique: Scale and Image Pyramids
7.5.1 The Gaussian Pyramid
7.5.2 Applications of Scaled Representations
7.5.3 Scale Space
7.6 Discussion
7.6.1 Real Imaging Systems vs Shift-Invariant Linear Systems
7.6.2 Scale
8 EDGE DETECTION
8.1 Estimating Derivatives with Finite Differences
8.1.1 Differentiation and Noise
8.1.2 Laplacians and edges
8.2 Noise
8.2.1 Additive Stationary Gaussian Noise
8.3 Edges and Gradient-based Edge Detectors
8.3.1 Estimating Gradients
8.3.2 Choosing a Smoothing Filter
8.3.3 Why Smooth with a Gaussian?
8.3.4 Derivative of Gaussian Filters
8.3.5 Identifying Edge Points from Filter Outputs
8.4 Commentary
9 FILTERS AND FEATURES
9.1 Filters as Templates
9.1.1 Convolution as a Dot Product
9.1.2 Changing Basis
9.2 Human Vision: Filters and Primate Early Vision
9.2.1 The Visual Pathway
9.2.2 How the Visual Pathway is Studied
9.2.3 The Response of Retinal Cells
9.2.4 The Lateral Geniculate Nucleus
9.2.5 The Visual Cortex
9.2.6 A Model of Early Spatial Vision
9.3 Technique: Normalised Correlation and Finding Patterns
9.3.1 Controlling the Television by Finding Hands by Normalised Correlation
9.4 Corners and Orientation Representations
9.5 Advanced Smoothing Strategies and Non-linear Filters
9.5.1 More Noise Models
9.5.2 Robust Estimates
9.5.3 Median Filters
9.5.4 Mathematical morphology: erosion and dilation
9.5.5 Anisotropic Scaling
9.6 Commentary
10 TEXTURE 2
10.1 Representing Texture
10.1.1 Extracting Image Structure with Filter Banks
10.2 Analysis (and Synthesis) Using Oriented Pyramids
10.2.1 The Laplacian Pyramid
10.2.2 Oriented Pyramids
10.3 Application: Synthesizing Textures for Rendering
10.3.1 Homogeneity
10.3.2 Synthesis by Matching Histograms of Filter Responses
10.3.3 Synthesis by Sampling Conditional Densities of Filter Responses
10.3.4 Synthesis by Sampling Local Models
10.4 Shape from Texture: Planes and Isotropy
10.4.1 Recovering the Orientation of a Plane from an Isotropic Texture
10.4.2 Recovering the Orientation of a Plane from an Homogeneity Assumption
10.4.3 Shape from Texture for Curved Surfaces
10.5 Notes
10.5.1 Shape from Texture
IV EARLY VISION: MULTIPLE IMAGES
11 THE GEOMETRY OF MULTIPLE VIEWS
11.1 Two Views
11.1.1 Epipolar Geometry
11.1.2 The Calibrated Case
11.1.3 Small Motions
11.1.4 The Uncalibrated Case
11.1.5 Weak Calibration
11.2 Three Views
11.2.1 Trifocal Geometry
11.2.2 The Calibrated Case
11.2.3 The Uncalibrated Case
11.2.4 Estimation of the Trifocal Tensor
11.3 More View's
11.4 Notes
11.5 Assignments
12 STEREO PSIS
12.1 Reconstruction
12.1.1 Camera Cal ibration
12.1.2 Image Rectification
Human Vision: Stereopsis
12.2 Binocular Fusion
12.2.1 Correlation
12.2.2 Multi-Scale Edge Matching
12.2.3 Dynamic Programming
12.3 Using More Cameras
12.3.1 Trinocular Stereo
12.3.2 Multiple-Baseline Stereo
12.4 Notes
12.5 Assignments
13 AFFINE STRUCTURE FROM MOTION
13.1 Elements of Affine Geometry-
13.2 Affine Structure from Two Images
13.2.1 The Affine Structure-from-Motion Theorem
13.2.2 Rigidity and Metric Constraints
13.3 Affine Structure from Multiple Images
13.3.1 The Affine Structure of Affine Image Sequences
Technique: Singular Value Decomposition
13.3.2 A Factorization Approach to Affine Motion Analysis
13.4 From Affine to Euclidean Images
13.4.1 Euclidean Projection Models
13.4.2 From Affine to Euclidean Motion
13.5 Affine Motion Segmentation
13.5.1 The Reduced Echelon Form of the Data Matrix
13.5.2 The Shape Interaction Matrix
13.6 Notes
13.7 Assignments
14 PROJECTIVE STRUCTURE FROM MOTION
14.1 Elements of Projective Geometry
14.1.1 Projective Bases and Projective Coordinates
14.1.2 Projective Transformations
14.1.3 Affine and Projective Spaces
14.1.4 Hyperplanes and Duality
14.1.5 Cross-Ratios
14.1.6 Application: Parameterizing the Fundamental Matrix
14.2 Projective Scene Reconstruction from Two View’s
14.2.1 Analytical Scene Reconstruction
14.2.2 Geometric Scene Reconstruction
14.3 Motion Estimation from Two or Three Views
14.3.1 Motion Estimation from Fundamental Matrices
14.3.2 Motion Estimation from Trifocal Tensors
14.4 Motion Estimation from Multiple Views
14.4.1 A Factorization Approach to Projective Motion Analysis
14.4.2 Bundle Adjustment
14.5 From Projective to Euclidean Structure and Motion
14.5.1 Metric Upgrades from (Partial) Camera Calibration
14.5.2 Metric Upgrades from Minimal Assumptions
14.6 Notes
14.7 Assignments
V MID-LEVEL VISION
15 SEGMENTATION USING CLUSTERING METHODS
15.1 Human vision: Grouping and Gestalt
15.2 Applications: Shot Boundary Detection. Background Subtraction
and Skin Finding
15.2.1 Background Subtraction
15.2.2 Shot Boundary Detection
15.2.3 Finding Skin Using Image Colour
15.3 Image Segmentation by Clustering Pixels
15.3.1 Simple Clustering Methods
15.3.2 Segmentation Using Simple Clustering Methods
15.3.3 Clustering and Segmentation by K-means
15.4 Segmentation by Graph-Theoretic Clustering
15.4.1 Basic Graphs
15.4.2 The Overall Approach
15.4.3 Affinity Measures
15.4.4 Eigenvectors and Segmentation
15.4.5 Normalised Cuts
15.5 Discussion
16 FITTING
16.1 The Hough Transform
16.1.1 Fitting Lines with the Hough Transform
16.1.2 Practical Problems with the Hough Transform
16.2 Fitting Lines
16.2.1 Least Squares, Maximum Likelihood and Parameter Estimation
16.2.2 Which Point is on Which Line?
16.3 Fitting Curves
16.3.1 Implicit Curves
16.3.2 Parametric Curves
16.4 Fitting to the Outlines of Surfaces
16.4.1 Some Relations Between Surfaces and Outlines
16.4.2 Clustering to Form Symmetries
16.5 Discussion
17 SEGMENTATION AND FITTING USING PROBABILISTIC METHODS
17.1 Missing Data Problems, Fitting and Segmentation
17.1.1 Missing Data Problems
17.1.2 The EM Algorithm
17.1.3 Colour and Texture Segmentation with EM
17.1.4 Motion Segmentation and EM
17.1.5 The Number of Components
17.1.6 How Many Lines are There?
17.2 Robustness
17.2.1 Explicit Outliers
17.2.2 M-estimators
17.2.3 RANSAC
17.3 How Many are There?
17.3.1 Basic Ideas
17.3.2 AIC — An Information Criterion
17.3.3 Bayesian methods and Schwartz’ BIC
17.3.4 Description Length
17.3.5 Other Methods for Estimating Deviance
17.4 Discussion
18 TRACKING
18.1 Tracking as an Abstract Inference Problem
18.1.1 Independence Assumptions
18.1.2 Tracking as Inference
18.1.3 Overview
18.2 Linear Dynamic Models and the Kalman Filter
18.2.1 Linear Dynamic Models
18.2.2 Kalman Filtering
18.2.3 The Kalman Filter for a ID State Vector
18.2.4 The Kalman Update Equations for a General State Vector
18.2.5 Forward-Backward Smoothing
18.3 Non-Linear Dynamic Models
18.3.1 Unpleasant Properties of Non-Linear Dynamics
18.3.2 Difficulties with Likelihoods
18.4 Particle Filtering
18.4.1 Sampled Representations of Probability Distributions
18.4.2 The Simplest Particle Filter
18.4.3 A Workable Particle Filter
18.4.4 If’s, And’s and But’s — Practical Issues in Building Particle Filters
18.5 Data Association
18.5.1 Choosing the Nearest — Global Nearest Neighbours
18.5.2 Gating and Probabilistic Data Association
18.6 Applications and Examples
18.6.1 Vehicle Tracking
18.6.2 Finding and Tracking People
18.7 Discussion
II Appendix: The Extended Kalman Filter, or EKF
VI HIGH-LEVEL VISION
19 CORRESPONDENCE AND POSE CONSISTENCY
19.1 Initial Assumptions
19.1.1 Obtaining Hypotheses
19.2 Obtaining Hypotheses by Pose Consistency
19.2.1 Pose Consistency for Perspective Cameras
19.2.2 Affine and Projective Camera Models
19.2.3 Linear Combinations of Models
19.3 Obtaining Hypotheses by Pose Clustering
19.4 Obtaining Hypotheses Using Invariants
19.4.1 Invariants for Plane Figures
19.4.2 Geometric Hashing
19.4.3 Invariants and Indexing
19.5 Verification
19.5.1 Edge Proximity
19.5.2 Similarity in Texture, Pattern and Intensity
19.5.3 Example: Baves Factors and Verification
19.6 Application: Registration in Medical Imaging Systems
19.6.1 Imaging Modes
19.6.2 Applications of Registration
19.6.3 Geometric Hashing Techniques in Medical Imaging
19.7 Curved Surfaces and Alignment
19.8 Discussion
20 FINDING TEMPLATES USING CLASSIFIERS
20.1 Classifiers
20.1.1 Using Loss to Determine Decisions
20.1.2 Overview: Methods for Building Classifiers
20.1.3 Example: A Plug-in Classifier for Normal Class-conditional Densities
20.1.4 Example: A Non-Parametric Classifier using Nearest Neighbours
20.1.5 Estimating and Improving Performance
20.2 Building Classifiers from Class Histograms
20.2.1 Finding Skin Pixels using a Classifier
20.2.2 Face Finding Assuming Independent Template Responses
20.3 Feature Selection
20.3.1 Principal Component Analysis
20.3.2 Canonical Variates
20.4 Neural Networks
20.4.1 Key Ideas
20.4.2 Minimizing the Error
20.4.3 When to Stop Training
20.4.4 Finding Faces using Neural Networks
20.4.5 Convolutional Neural Nets
20.5 The Support Vector Machine
20.5.1 Support Vector Machines for Linearly Separable Datasets
20.5.2 Finding Pedestrians using Support Vector Machines
20.6 Conclusions
II Appendix: Support Vector Machines for Datasets that are not Linearly Separable
III Appendix: Using Support Vector Machines with Non-Linear Kernels
21 RECOGNITION BY RELATIONS BETWEEN TEMPLATES
21.1 Finding Objects by Voting on Relations between Templates
21.1.1 Describing Image Patches
21.1.2 Voting and a Simple Generative Model
21.1.3 Probabilistic Models for Voting
21.1.4 Voting on Relations
21.1.5 Voting and 3D Objects
21.2 Relational Reasoning using Probabilistic Models and Search
21.2.1 Correspondence and Search
21.2.2 Example: Finding Faces
21.3 Using Classifiers to Prune Search
21.3.1 Identifying Acceptable Assemblies Using Projected Classifiers
21.3.2 Example: Finding People and Horses Using Spatial Relations
21.4 Technique: Hidden Markov Models
21.4.1 Formal Matters
21.4.2 Computing with Hidden Markov Models
21.4.3 Varieties of HMM's
21.5 Application: Hidden Markov Models and Sign Language Understandin
21.6 Application: Finding People with Hidden Markov Models
21.7 Frames and Probability Models
21.7.1 Representing Coordinate Frames Explicitlv in a Probability Model
21.7.2 Using a Probability Model to Predict Feature Positions
21.7.3 Building Probability Models that are Frame-Invariant.
21.7.4 Example: Finding Faces Using Frame Invariance
21.8 Conclusions
22 ASPECT GRAPHS
22.1 Differential Geometry and Visual Events
22.1.1 The Geometry of the Gauss Map
22.1.2 Asymptotic Curves
22.1.3 The Asymptotic Spherical Map
22.1.4 Local Visual Events
22.1.5 The Bitangent Ray Manifold
22.1.6 Multilocal Visual Events
22.1.7 Remarks
22.2 Computing the Aspect Graph
22.2.1 Step 1: Tracing Visual Events
22.2.2 Step 2: Constructing the Regions
22.2.3 Remaining Steps of the Algorithm
22.2.4 An Example
22.3 Aspect Graphs and Object Recognition
22.4 Notes
22.5 Assignments
VII APPLICATIONS AND TOPICS
23 RANGE DATA
23.1 Active Range Sensors
23.2 Range Data Segmentation
Technique: Analytical Differential Geometry
23.2.1 Finding Step and Roof Edges in Range Images
23.2.2 Segmenting Range Images into Planar Regions
23.3 Range Image Registration and Model Construction
Technique: Quaternions
23.3.1 Registering Range Images Using the Iterative Closest-Point Method
23.3.2 Fusing Multiple Range Images
23.4 Object Recognition
23.4.1 Matching Piecewise-Planar Surfaces Using Interpretation Trees
23.4.2 Matching Free-Form Surfaces Using Spin Images
23.5 Notes
23.6 Assignments
24 APPLICATION: FINDING IN DIGITAL LIBRARIES
24.1 Background
24.1.1 What do users want?
24.1.2 What can tools do?
24.2 Appearance
24.2.1 Histograms and correlograms
24.2.2 Textures and textures of textures
24.3 Finding
24.3.1 Annotation and segmentation
24.3.2 Template matching
24.3.3 Shape and correspondence
24.4 Video
24.5 Discussion
25 APPLICATION: IMAGE-BASED RENDERING
25.1 Constructing 3D Models from Image Sequences
25.1.1 Scene Modeling from Registered Images
25.1.2 Scene Modeling from Unregistered Images
25.2 Transfer-Based Approaches to Image-Based Rendering
25.2.1 Affine View Synthesis
25.2.2 Euclidean View Synthesis
25.3 The Light Field
25.4 Notes
25.5 Assignments