Teaching code

4S010673

Credits

12

Coordinator

Marco Cristani

Language

English

Scientific Disciplinary Sector (SSD)

ING-INF/05 - INFORMATION PROCESSING SYSTEMS

Moodle Seminars 0

The teaching is organized as follows:

Learning objectives

The course aims to provide the theoretical foundations and describe the main methodologies relating to the area of machine learning, together with the most recent techniques of deep learning. In particular, the course will deal with describing the methods of analysis, recognition and automatic classification of data of any type, typically called patterns. These disciplines are at the base, are used, and often complement many other disciplines and application areas of wide diffusion, such as computational vision, robotics, image processing, data mining, analysis and interpretation of medical and biological data, bioinformatics, biometrics, video surveillance, forecasting. More precisely, the methodologies that will be introduced in the course are often an integral part of the application areas mentioned above, and constitute the "intelligent" part of it with the final aim of understanding (classifying, recognizing, analyzing) the data coming from the process of interest (be they signals, images, strings, categorical, or other types). Starting from the type of data measured, the entire analysis pipeline will be considered, such as the extraction and selection of characteristics, supervised and unsupervised machine learning methods, parametric and non-parametric analysis techniques, and validation protocols, together with visualization necessary for understanding deep learning systems. At the laboratory level, real case studies and not just academic benchmarks will be presented, addressed with appropriate programming tools. In conclusion, the course aims to provide the student with a set of theoretical foundations and algorithmic tools to address the problems that may be encountered in strategic and innovative industrial sectors such as those involving the processing of large amounts of data (big data), multimedia, visual inspection of products, automation and forecasting.

Prerequisites and basic notions

Basic knowledge of probability and statistics is required.

Program

*MACHINE LEARNING MODULE*:
-Course introduction
-Different modules of a classifier
-The pattern recognition pipeline
-Different types of learning
-Bayes’ law
-Gaussian identities
-Discriminant functions: discriminative vs generative
-Linear projections
-Principal Component Analysis (PCA)
-Eigenfaces
-Bag-of-words; autoencoders
-Generative modeling: maximum likelihood estimation (MLE)
-Expectation–Maximization (EM)
-Parzen windows: non-parametric density estimation
-Mean shift; Monte Carlo dynamic density estimation
-Support Vector Machines (SVM)
-Unsupervised learning: taxonomy
-Unsupervised learning: validation techniques

*DEEP LEARNING MODULE*
-History of deep learning
-Applications of deep learning
-Linear regression as basis for DL models
-Bias derivative in linear regression
-Data normalization: min–max, z-norm
-Regularization in deep networks: Lasso and Ridge
-Fully connected networks; backpropagation (part I)
-Backpropagation exercise; dropout as Bayesian approximation
-Backpropagation on a fully connected network
-Convolutional layers (CNN)
-Nonlinearities in CNNs
-CNN interpretability
-NN visualization methods: CAM, Grad-CAM, t-SNE
-ResNet and residual blocks
-Batch Normalization and 1×1 convolutions
-Recursive neural networks
-R-CNN, Fast R-CNN, Faster R-CNN
-Transformer models
-From self-supervised to diffusion models

Bibliography

Didactic methods

-Lectures: Theoretical presentation with examples and exercises on the board.
-ML Lab (MATLAB): Implementation of classic pipelines (PCA, SVM, EM, Parzen, validation); guided scripts on provided datasets.
-DL Lab (Python): Implementation of networks (FCN, CNN/ResNet, Transformer), backprop, and BN in notebooks; use of standard libraries.
-Structure: Each topic is covered first in the classroom, then translated into practice in the lab. -Materials: Slides, notebooks, and scripts shared on Moodle

Learning assessment procedures

An oral exam consisting of a discussion of the theory covered in class and a discussion of a project assigned previously in class, approximately 15 days before the end of class, separately for both the Machine Learning and Deep Learning modules. The oral exams (theory and project) will take place on the same day.

Evaluation criteria

For the theoretical part, the grade will depend on the accuracy of the answers and the ability to reproduce the proofs seen in class. For the laboratory part, the grade will depend on the classification capabilities achieved by the classifier (with classification accuracy measures varying from problem to problem), the statistical confidence levels provided, and the theoretical motivation that led the student to choose a particular programming technique.

Criteria for the composition of the final grade

For each module (Machine Learning and Deep Learning), a maximum of 16 points is awarded for the theoretical part and a maximum of 16 points for the project part. The final grade for the entire course will be the average of the overall grades for each module.

Exam language

inglese