Fiche de révision : Advanced Image Recognition and Classification

📋 Course Outline

1. Image matching techniques
2. Feature descriptors
3. Image classification basics
4. Learning paradigms
5. Linear classifiers
6. Support vector machines
7. Ensemble methods
8. Object detection
9. Performance metrics
10. Kernel trick in SVM

📖 1. Image matching techniques

🔑 Key Concepts & Definitions

• Interest point detection: The process of identifying salient points in an image that are invariant to transformations, used as keypoints for matching across images. These points are typically distinctive and repeatable, facilitating reliable correspondence.

• Harris detector: An interest point detection method introduced by Harris and Stephens (1988), which identifies corners by analyzing the local autocorrelation of image intensities. It computes a response function based on the eigenvalues of the second-moment matrix, highlighting points with significant intensity variation in multiple directions.

• Scale-adapted Harris detector: An extension of the Harris detector that incorporates scale-space analysis, enabling the detection of interest points at multiple scales. This approach adjusts the detection process to be robust to changes in object size, often by applying the Harris detector across a scale pyramid.

• Laplacian-based detector: An interest point detection technique that uses the Laplacian of Gaussian (LoG) or Difference of Gaussians (DoG) to identify blob-like structures in images. It detects points where the Laplacian response is extremal, which are typically stable across scales and transformations.

• Scale-invariant feature transform (SIFT): A robust feature detection and description method developed by Lowe (2004). SIFT detects keypoints that are invariant to scale and rotation by identifying extrema in a scale-space constructed via Gaussian blurring and difference-of-Gaussians. It then computes distinctive descriptors for matching.

• Matching algorithm: The procedure that establishes correspondences between interest points in different images, typically by comparing feature descriptors using metrics like Euclidean distance, and applying strategies such as nearest neighbor search to find the best matches.

📝 Essential Points

• Interest point detection aims to find repeatable features that are robust to scale, rotation, and illumination changes, crucial for reliable image matching (Harris and Stephens, 1988; Lowe, 2004).

• The Harris detector is computationally efficient but primarily detects corners at a fixed scale; thus, it is often combined with scale-space techniques for scale invariance.

• Scale-adapted Harris detectors extend the original method by applying the Harris response across multiple scales, improving detection robustness to size variations.

• Laplacian-based detectors, such as LoG and DoG, excel at blob detection and are inherently scale-invariant, making them suitable for detecting features across different image resolutions.

• SIFT combines interest point detection with a robust descriptor, enabling high matching accuracy even under significant transformations. Its keypoints are selected as local extrema in the scale-space, and descriptors are formed from gradient histograms around each keypoint.

• Matching algorithms compare feature descriptors between images, often using nearest neighbor search in descriptor space, and may include ratio tests or geometric verification to filter false matches.

💡 Key Takeaway

Interest point detection methods like Harris, scale-adapted Harris, Laplacian-based detectors, and SIFT are fundamental for extracting stable, distinctive features in images. These features, coupled with effective matching algorithms, enable reliable image correspondence and are essential for tasks like object recognition and 3D reconstruction.

📖 2. Feature descriptors

🔑 Key Concepts & Definitions

• Basic features: Hand-crafted features that involve simple image properties such as pixel intensities, histograms, or gradients, used to describe local or global image characteristics.

• SIFT features: Scale-Invariant Feature Transform (Lowe, 2004): a hand-crafted feature that detects interest points and computes descriptors invariant to scale, rotation, and illumination changes, capturing local image structure.

• Speeded-Up Robust Features (SURF): An efficient hand-crafted feature (Bay et al., 2008) that detects interest points using Haar wavelets and computes descriptors based on gradient information, optimized for speed and robustness.

• Hand-crafted features: Features explicitly designed by humans based on domain knowledge, such as pixel intensities, histograms, or gradients, to encode image information for tasks like classification or matching.

• Learned features: Features automatically learned from data, typically via convolutional neural networks (CNNs), where the feature extraction process is integrated into the training of the model, enabling adaptive and hierarchical feature representation.

• Feature extraction transforming images into low-dimensional vectors: The process of converting raw images into compact, discriminative vectors (feature descriptors) that facilitate comparison and classification, reducing computational complexity and enhancing robustness.

📝 Essential Points

• Hand-crafted features like pixel intensities, histograms, and gradients are designed to capture specific image properties and are often used in traditional computer vision tasks. Examples include pixel intensity histograms and gradient-based descriptors.

• SIFT features, introduced by Lowe (2004), are particularly notable for their invariance to scale, rotation, and illumination, making them highly effective for matching and recognition tasks across varying conditions.

• SURF, developed by Bay et al. (2008), improves upon earlier interest point detectors by offering faster computation while maintaining robustness, primarily through the use of Haar wavelet responses and gradient-based descriptors.

• Learned features, especially those derived from convolutional neural networks, enable the model to automatically discover the most relevant features for a given task, often outperforming hand-crafted features in complex scenarios.

• The transformation of images into low-dimensional vectors via feature extraction is crucial for efficient classification and matching, as it simplifies the data while preserving discriminative information.

💡 Key Takeaway

Feature descriptors—whether hand-crafted like SIFT and SURF or learned through CNNs—are essential for transforming raw images into meaningful, low-dimensional vectors that facilitate robust and efficient image classification and matching.

📖 3. Image classification basics

🔑 Key Concepts & Definitions

• Image classification task definition: The process of assigning a label or category to an entire image based on its visual content, often involving estimating class scores that indicate the likelihood of each class (source: Giacomo Tarroni).

• Class label and class scores: The class label is the categorical identifier assigned to an image (e.g., "cat"). Class scores are numerical values representing the confidence or probability that the image belongs to each class, which can be used to determine the final label (source: Giacomo Tarroni).

• Common datasets (MNIST, ImageNet): Standardized collections of labeled images used to train and evaluate image classification algorithms. MNIST contains ~70,000 handwritten digit images (28x28 pixels, 10 classes), while ImageNet includes ~15 million natural images across 1,000 classes with a size of 256x256x3 RGB (source: Giacomo Tarroni).

• Machine learning approach for classification: A method where input images are transformed into feature descriptors or vectors, which are then used to train models to predict class labels. This approach relies on learning from labeled datasets to generalize to unseen images (source: Giacomo Tarroni).

• Feature extraction and classification pipeline: The process of transforming raw images into low-dimensional, discriminative feature vectors, followed by training classifiers (e.g., SVM, neural networks) to assign class labels based on these features. This pipeline enables effective comparison and decision-making (source: Giacomo Tarroni).

• Examples of classifiers: Algorithms such as K-nearest neighbors (KNN), linear classifiers (including support vector machines), and neural networks, which are trained on feature vectors to perform image classification tasks (source: Giacomo Tarroni).

📝 Essential Points

• Image classification involves assigning a class label based on the visual content, often using class scores to quantify confidence (source: Giacomo Tarroni).

• Datasets like MNIST and ImageNet serve as benchmarks for training and testing classification models, with MNIST focusing on handwritten digits and ImageNet on diverse natural images (source: Giacomo Tarroni).

• The machine learning approach transforms images into feature vectors, which are then used to train models that can predict class labels for new images, emphasizing the importance of discriminative and invariant features (source: Giacomo Tarroni).

• The feature extraction step can involve hand-crafted features or learned features via neural networks, which are then fed into classifiers such as SVMs or neural networks for decision-making (source: Giacomo Tarroni).

• The choice of classifier impacts the accuracy and robustness of the image classification system, with examples including KNN, linear classifiers, and deep neural networks (source: Giacomo Tarroni).

💡 Key Takeaway

Image classification relies on transforming raw images into meaningful feature representations and training models to accurately assign class labels, with datasets like MNIST and ImageNet providing essential benchmarks for progress in the field.

📖 4. Learning paradigms

🔑 Key Concepts & Definitions

• Supervised learning: A learning paradigm where the training data includes input-output pairs, with labels available for each example, enabling the model to learn the mapping from inputs to known outputs (source content).
• Unsupervised learning: A paradigm where training data consists only of input data without labels, and the goal is to find inherent structures or groupings within the data, such as clustering (source content).
• Semi-supervised learning: A hybrid approach where only part of the training data is labeled, and the model leverages both labeled and unlabeled data to improve learning efficiency and accuracy (source content).
• Training/test data split: The process of dividing the dataset into separate subsets for training the model and evaluating its performance, typically to prevent overfitting and assess generalization (source content).
• Hyper-parameter tuning and validation: The procedure of selecting optimal high-level parameters (hyper-parameters) of the model by evaluating performance on a validation set, avoiding overfitting to the training data (source content).
• K-fold cross-validation: A validation technique where the dataset is partitioned into K subsets; the model is trained on K-1 folds and validated on the remaining fold, repeated K times, and the results are averaged to estimate performance (source content).

📝 Essential Points

• Supervised learning requires labeled datasets, making it suitable for tasks like image classification where class labels are known (source content).
• Unsupervised learning is used when labels are unavailable, often for clustering or dimensionality reduction, to discover data structure (source content).
• Semi-supervised learning is particularly useful when obtaining labels is costly or time-consuming, allowing models to learn from limited labeled data combined with abundant unlabeled data (source content).
• Proper data splitting into training and test sets is crucial for unbiased evaluation of model performance; typically, a common split is 90% training and 10% testing (source content).
• Hyper-parameter tuning involves adjusting parameters like regularization strength or kernel parameters, often using a validation set or K-fold cross-validation to avoid overfitting on the test data (source content).
• K-fold cross-validation enhances model robustness by averaging performance over multiple train-test splits, especially beneficial for small datasets (source content).

💡 Key Takeaway

Learning paradigms define how models are trained and validated based on the availability of labels, with techniques like data splitting and cross-validation ensuring reliable performance assessment and hyper-parameter optimization.

📖 5. Linear classifiers

🔑 Key Concepts & Definitions

• Linear classifier decision boundary: A boundary that separates different classes in the feature space, defined as a hyperplane where the classifier's decision changes from one class to another. It is represented by a linear equation involving weights and bias.

• Hyperplane equation in 2D and D dimensions: In 2D, the hyperplane (decision boundary) is expressed as $w_1 x_1 + w_2 x_2 + b = 0$. In D dimensions, it generalizes to $\mathbf{w} \cdot \mathbf{x} + b = 0$, where $\mathbf{w}$ is the weight vector and $\mathbf{x}$ is the feature vector.

• Parameters $\mathbf{w}$ (weights) and $b$ (bias): $\mathbf{w}$ is a vector of weights that determine the orientation of the hyperplane, while $b$ shifts the hyperplane's position relative to the origin. These parameters define the decision boundary.

• Decision rule based on hyperplane: Class assignment is made by evaluating the sign of $\mathbf{w} \cdot \mathbf{x} + b$. If the result is ≥ 0, the data point belongs to one class; if < 0, to the other.

• Training linear classifiers by finding $\mathbf{w}$ and $b$: The process involves optimizing these parameters to best separate the classes, often by maximizing the margin (see support vector machines) or minimizing classification errors, depending on the specific method.

📝 Essential Points

• The decision boundary of a linear classifier is a hyperplane described by the equation $\mathbf{w} \cdot \mathbf{x} + b = 0$ in D-dimensional space, with the normal vector $\mathbf{w}$ perpendicular to the hyperplane.

• In 2D, this hyperplane simplifies to a line $w_1 x_1 + w_2 x_2 + b = 0$, which divides the plane into two regions corresponding to different classes.

• The parameters $\mathbf{w}$ and $b$ are learned from training data by solving an optimization problem that aims to find the hyperplane that best separates the classes, such as maximizing the margin in support vector machines.

• The decision rule relies on the sign of the linear function $\mathbf{w} \cdot \mathbf{x} + b$, enabling straightforward classification of new data points based on their position relative to the hyperplane.

• Proper training involves adjusting $\mathbf{w}$ and $b$ to minimize misclassification errors or maximize the margin, which enhances the classifier's robustness and generalization.

💡 Key Takeaway

A linear classifier separates data into classes using a hyperplane defined by weights and bias, with the decision rule based on the sign of a linear function. Training involves finding the optimal $\mathbf{w}$ and $b$ to achieve the best possible separation.

📖 6. Support vector machines

🔑 Key Concepts & Definitions

• Maximum margin hyperplane: The decision boundary that maximizes the distance (margin) between itself and the closest data points of each class, ensuring optimal separation. Support vector machines (SVMs) aim to find this hyperplane (**********).
• Support vectors: The subset of training data points that lie closest to the decision boundary and directly influence the position and orientation of the hyperplane. Only support vectors satisfy the constraints w·x + b = ±1.
• Constraints for support vectors: For each support vector xᵢ with label yᵢ, the hyperplane parameters w and b must satisfy yᵢ(w·xᵢ + b) = 1 (see source content). These constraints ensure support vectors lie on the margin boundaries.
• Margin maximization objective: The goal of SVMs is to maximize the margin m = 2 / ||w||, which is equivalent to minimizing ||w||² while respecting the constraints for support vectors. This leads to a convex quadratic optimization problem.
• Slack variables: Introduced by Andrew Zisserman to handle non-linearly separable data, slack variables ξᵢ ≥ 0 allow some data points to violate the margin constraints, enabling a soft margin approach. The constraints become yᵢ(w·xᵢ + b) ≥ 1 - ξᵢ.
• Hinge loss function and regularization term: The SVM optimization combines the hinge loss max(0, 1 - yᵢ(w·xᵢ + b)) with a regularization term C·∑ξᵢ to penalize margin violations, balancing margin maximization and classification errors.

📝 Essential Points

• The maximum margin hyperplane is found by solving a convex quadratic optimization problem that maximizes the margin 2 / ||w|| while satisfying the constraints yᵢ(w·xᵢ + b) ≥ 1 for support vectors.
• In cases where data is not linearly separable, slack variables ξᵢ are introduced to permit some violations, controlled by the regularization parameter C. A larger C enforces stricter margin adherence, while a smaller C allows more violations, leading to a soft margin.
• The hinge loss function max(0, 1 - yᵢ(w·xᵢ + b)) quantifies the penalty for misclassification or margin violations, and the overall optimization minimizes the sum of hinge loss and regularization C·∑ξᵢ.
• The convexity of the optimization problem guarantees a global minimum, making the solution reliable. Techniques like Lagrangian duality and quadratic programming are used for efficient solving.
• The support vectors are the only data points that determine the hyperplane; the model's complexity depends on their number and position.

💡 Key Takeaway

Support Vector Machines aim to find the optimal separating hyperplane with the largest margin, using support vectors and slack variables to handle both linearly separable and non-separable data, balancing margin maximization with classification errors through hinge loss and regularization.

📖 7. Ensemble methods

🔑 Key Concepts & Definitions

• Committees: An ensemble approach where multiple classifiers are trained independently on the same data, and their predictions are combined by averaging or voting to improve overall accuracy. This method leverages diversity among classifiers to reduce variance and overfitting.

• Boosting: A sequential ensemble technique that trains classifiers iteratively, where each new classifier focuses on the errors of the previous ones by assigning higher weights to misclassified samples. The final prediction is a weighted majority vote of all classifiers, often producing strong performance even with weak learners. ADABoost (see source) is a prominent example.

• Cascading classifiers: An ensemble strategy that concatenates multiple classifiers in a sequence, where each classifier filters out negatives, passing only potential positives to the next stage. This approach is designed for rapid detection, especially in object detection tasks, by focusing computational resources on difficult samples.

📝 Essential Points

• Ensemble methods aim to improve performance by combining multiple classifiers, reducing the risk of overfitting and increasing robustness (see "Ensemble of linear classifiers" and "Boosting" sections).
• Committees are simple and effective, often created via bootstrapping, which involves training classifiers on different random samples of data, then aggregating their predictions to enhance generalization.
• Boosting iteratively trains classifiers, emphasizing misclassified samples through weighted datasets, and combines them via weighted majority voting. This process allows weak learners, like stumps, to form a powerful classifier (see AdaBoost example).
• Cascading classifiers are particularly useful in real-time object detection, where early stages quickly discard obvious negatives, and later stages focus on challenging positives, optimizing speed and accuracy (see Viola-Jones detector).
• These ensemble techniques are widely used in image classification and object detection, often leading to higher accuracy and faster inference compared to single classifiers.

💡 Key Takeaway

Ensemble methods such as committees, boosting, and cascading classifiers combine multiple models to enhance accuracy, robustness, and efficiency, making them essential tools in advanced image classification and object detection tasks.

📖 8. Object detection

🔑 Key Concepts & Definitions

• Viola-Jones object detector (2001): A real-time object detection framework that uses a sliding window approach combined with Haar wavelet features, AdaBoost for feature selection, and a cascade of classifiers to efficiently detect objects such as faces. It is based on simple features and rapid classification techniques, enabling fast detection in images.

• Object detection task definition: The process of identifying and locating multiple objects within an image, assigning each object a class label and bounding box coordinates (x, y, width, height). Unlike classification, which labels the entire image, object detection involves both classification and localization.

📝 Essential Points

• The Viola-Jones detector applies a sliding window over the image, maintaining an aspect ratio similar to the object of interest, to scan for objects like faces. For each window position, a large set of Haar wavelet features are computed, capturing intensity differences that highlight object parts such as eyes or cheeks.

• Features are concatenated into a feature descriptor, which is then classified using a cascade of linear classifiers (stumps) trained with AdaBoost. This cascade structure allows rapid rejection of non-object regions, focusing computational resources on promising areas.

• The training involves a labeled dataset of image crops, with positive samples containing the object and negatives without. The cascade is trained to balance detection accuracy and speed, making the detector suitable for real-time applications.

• OpenCV provides pre-trained Viola-Jones models (e.g., haarcascade_frontalface_default.xml) that can be used directly for face detection, or trained custom models for specific object detection tasks.

• Other feature descriptors like Histograms of Oriented Gradients (HOGs) and Local Binary Patterns (LBPs) can also be used for object detection, often in combination with classifiers such as SVMs or neural networks.

💡 Key Takeaway

The Viola-Jones object detector is a pioneering, efficient framework that combines simple Haar features, boosting, and cascading classifiers to enable fast, real-time object detection, especially for faces, by effectively balancing accuracy and computational speed.

📖 9. Performance metrics

🔑 Key Concepts & Definitions

• True Positives (TP): The number of correctly identified positive cases, i.e., images of the target class (e.g., cats) that are correctly labeled as such.

• True Negatives (TN): The number of correctly identified negative cases, i.e., images not belonging to the target class that are correctly labeled as non-target.

• False Positives (FP): The number of incorrect positive identifications, i.e., images of non-target classes wrongly labeled as the target class.

• False Negatives (FN): The number of missed positive cases, i.e., images of the target class wrongly labeled as non-target.

• Sensitivity (Recall): (see source content): The proportion of actual positives correctly identified, calculated as TP / (TP + FN). It measures the classifier’s ability to detect positive cases.

• Specificity: (see source content): The proportion of actual negatives correctly identified, calculated as TN / (TN + FP). It indicates how well the classifier avoids false alarms.

📝 Essential Points

• Performance metrics are derived from the confusion matrix, which summarizes the classifier's predictions against true labels.

• Sensitivity (Recall) emphasizes the classifier’s ability to detect positive instances, crucial in applications where missing positives is costly.

• Specificity focuses on correctly rejecting negatives, important in scenarios where false alarms have high consequences.

• These metrics are fundamental for evaluating and comparing classifiers, especially in imbalanced datasets where accuracy alone can be misleading.

• The source highlights the importance of understanding TP, TN, FP, FN to interpret metrics like sensitivity and specificity accurately (see source content).

💡 Key Takeaway

Performance metrics such as sensitivity and specificity, based on true positives, true negatives, false positives, and false negatives, provide critical insights into a classifier’s effectiveness in distinguishing between classes and are essential for comprehensive evaluation.

📖 10. Kernel trick in SVM

🔑 Key Concepts & Definitions

• Feature transformation to higher-dimensional space: The process of mapping original input features into a space with more dimensions, often to make data linearly separable (see source content on feature transformation). This transformation allows complex data distributions to be separated by a hyperplane in the new space.

• Kernel trick concept in SVM: A method that enables the computation of inner products in a high-dimensional feature space without explicitly performing the transformation. Instead, a kernel function directly computes the dot product between the transformed features, making the process computationally efficient (see source content on kernel functions).

• Mapping original features to enable linear separability in transformed space: The technique of transforming data into a higher-dimensional space where a linear hyperplane can effectively separate classes that are not separable in the original space. This mapping is implicitly performed via kernel functions, avoiding explicit computation of the transformation (see source content on feature transformation and kernel trick).

📝 Essential Points

• The kernel trick allows SVMs to operate in a high-dimensional feature space without explicitly computing the transformation 𝜱(𝒙), by using kernel functions 𝑘(𝒙ᵢ, 𝒙ⱼ) that compute the inner product 𝜱(𝒙ᵢ) · 𝜱(𝒙ⱼ). This makes it feasible to handle non-linear data distributions efficiently.

• Common kernel functions include the linear kernel (𝑘(𝒙ᵢ, 𝒙ⱼ) = 𝒙ᵢ · 𝒙ⱼ), the polynomial kernel (𝑘(𝒙ᵢ, 𝒙ⱼ) = (1 + 𝒙ᵢ · 𝒙ⱼ)ᵈ), and the Gaussian (RBF) kernel (𝑘(𝒙ᵢ, 𝒙ⱼ) = exp(−γ ||𝒙ᵢ − 𝒙ⱼ||²)). These kernels implicitly perform the feature transformation to higher-dimensional spaces.

• The mapping to higher-dimensional space (see source content) is crucial for enabling linear separability in cases where data is not linearly separable in the original feature space, thus allowing SVMs to find a separating hyperplane in a transformed space.

• The advantage of the kernel trick is that it avoids explicit computation of the high-dimensional features, reducing computational complexity and enabling the use of complex, non-linear decision boundaries.

💡 Key Takeaway

The kernel trick in SVMs leverages kernel functions to implicitly map data into higher-dimensional spaces, enabling linear separation of complex data distributions efficiently without explicitly performing the transformation.

📅 Key Dates

(OMITTED: No significant dates provided in the content)

📊 Synthesis Tables

⚠️ Common Pitfalls & Confusions

1. Confusing Harris detector with Laplacian-based detectors; Harris is corner-focused, Laplacian detects blobs.
2. Assuming SIFT is invariant to all transformations; it is robust but not invariant to extreme distortions.
3. Believing hand-crafted features always outperform learned features; CNN-based features often outperform traditional methods in complex tasks.
4. Misunderstanding the difference between interest point detection and feature description; detection finds keypoints, descriptors encode them.
5. Overlooking the importance of scale-space in SIFT and Laplacian detectors for scale invariance.
6. Assuming linear classifiers are sufficient for complex image data; non-linear models or kernels often needed.
7. Misinterpreting the kernel trick as a way to explicitly compute high-dimensional mappings; it implicitly computes inner products in feature space.

✅ Exam Checklist

• Know Harris and Stephens (1988) method for interest point detection.
• Understand scale-adapted Harris and Laplacian-based detectors (LoG, DoG) for scale invariance.
• Describe the SIFT algorithm by Lowe (2004), including keypoint detection and descriptor computation.
• Differentiate between hand-crafted features (SIFT, SURF) and learned features (CNN-based).
• Explain the process of transforming images into low-dimensional feature vectors for classification.
• Define the image classification task and distinguish class labels from class scores.
• Know the standard datasets MNIST and ImageNet, including their characteristics.
• Describe the typical machine learning pipeline: feature extraction followed by classifier training.
• Understand the basics of linear classifiers and their limitations.
• Know the concept of Support Vector Machines (Cortes & Vapnik, 1995), including the max-margin principle.
• Explain the kernel trick in SVMs as introduced by Schölkopf & Smola (2002), and its purpose.
• Be familiar with ensemble methods like bagging and boosting, and their role in improving model performance.