The teaching is organized as follows:

Learning objectives

The course introduces students to the fundamental theories and concepts of human-computer interaction (HCI), which is an interdisciplinary field that draws from cognitive psychology, computer science, and design. HCI aims to provide both theoretical understanding and hands-on experience in key aspects of human perception, cognition, and learning as they relate to interface design, implementation, and evaluation. Covered topics include interface design, usability evaluation, universal design, and multimodal interfaces (touch, vision, natural language, audio). Special emphasis will be placed on equipping students with foundational knowledge, methodologies, and tools necessary for designing, implementing, and evaluating computer systems capable of capturing, representing, and automatically analyzing user behavior. This encompasses various forms of non-verbal communication, including gestures, movements, facial expressions, and speech. Moreover, students will learn strategies for effectively interacting with users by providing multisensory feedback, utilizing elements such as images, sounds, and control of actuators.

At the end of the course, students will:
- Understand the rationale behind utilizing multimodal interactive systems for specific applications, comprehend the logical architectures defining the main components of such systems, grasp the design and development guidelines for multimodal interactive systems, and recognize the potential application areas for their successful deployment.
- Familiarize themselves with the key devices for capturing user behaviour data, comprehend their functionality, and discern appropriate usage scenarios.
- Acquire knowledge of essential techniques for representing and automatically analysing user behaviour, including those that process data from multiple sensor devices across various sensory channels.
-Demonstrate proficiency in designing and implementing major components of a multimodal interactive system using the development tools introduced in lectures and practical sessions throughout the course.

Program

Theory
Introduction: motivation, aim of the course, professional perspectives, open issues, description of course program and method of exam
Foundation of HCI: human factors, interaction design, usability, gaming and gamification
Visual interaction: camera calibration, structure and motion
Nonverbal behavior in communication: types of nonverbal behavior (facial expressions, gestures, posture, eye gaze), data collection methods, tools and software for nonverbal behavior analysis, annotation tools, e.g., ELAN
Automated analysis of body: movement, gestures, facial expressions, and speech. Data capturing techniques, extracting features, and automatic analysis
Social artificial intelligence: example applications, social psychology, organizational psychology, and social robotics.
Affective computing: theories of emotion, emotion recognition systems, applications of emotion recognition in HCI.
Integration of multimodal nonverbal cues: fusion techniques, e.g., late and early fusion.

Laboratory
Deep Image matching : Python implementation of feature detection and matching
3D Model reconstruction: Structure and motion with Zephyr
Camera pose estimation : C# implementation of Fiore’s method
3D graphics: modelling and rendering in Unity
Model-based AR: implementation of the full AR pipeline integrating python code and Unity
Advanced aspects: deep camera pose estimation, model recognition

Learning assessment procedures

Oral exam and evaluation of lab activity