«AI and ML methods for Multimodal Learning Analytics»
Kshitij SharmaNorwegian University os Science and Technology
Abstract
In this talk, I will present through different multimodal learning analytics case studies, how we can use the classical and advanced methods from artificial intelligence and machine learning to predict the students’ performance and experiences. I will also show how we can use the AI and ML pipelines to achieve certain level of generalizability from the multimodal data. I will show a few examples of explanability attempts in such methods. I will finally present a new method to design artificial agents to support and augment human learning using multimodal data and AI.
Bio
Kshitij Sharma is an Associate Professor at Norwegian Institute of Science and Technology at Trondheim, and the Academic Director of AI Transparency Institute. His background is in the area of Human-Computer Interaction and Collaborative/cooperative learning. In particular, his doctoral work was in the area of using multimodal data (EEG, eye-tracking, facial expressions, audio, dialogues, blood pressure, skin conductance, heart rate) to explain the differences between and predict, experts and novice groups; good and poor students; functional and non-functional groups. The main context for the application of his research has been education. His research interests are primarily in the area of Applied Machine Learning, Artificial Intelligence, and Human-Computer Interaction (HCI) with a heavy emphasis on groups’ behavior and physiological data such as eye-tracking, EEG, facial expressions (theoretical and practical methods in digital interaction). He seeks to understand relations between users’ data (EEG, eye-tracking, system log data, users’ actions) and the profile of the user (expertise, motivation, strategy, performance) based on empirical experimentation (controlled experiments) and mixed methods analysis (utilizing a multitude of digital technologies). The knowledge gained from these studies is then used to provide feedback to the group or adapt to the needs of the group in a proactive manner. For this effort, in his studies, he has combined eye-tracking and users’ actions to provide more comprehensive results through data science, statistics, and machine learning practices.
