Automatically Extracting Audiovisual Features to Analyze Videos in Educational Research
Affiliation: TU Darmstadt
External Partner: Graduate School Teaching & Learning Processes, Universität Koblenz-Landau
Financial Support by the Funding Agency : Dec 2015 – Nov 2017
A significant amount of time in research is spent to pre-process data for subsequent qualitative analysis. This is not different for researchers in the educational domain, especially if their data contains video recordings of classroom situations. In that case, the preparation phase typically involves speech transcription, videos sampling, format conversion and other time-intense tasks. The manual process might involve subjective analysis, such as judging students due to their (potentially problematic) personality, or rating the teachers’ interaction with the students in the classroom. Like that, the initial analysis creates a bottleneck for scaling the research task as the amount of data increases and also potentially introduces a bias in the subsequent analysis of the data.
The main goal of this project is to bootstrap the initial analysis of video recordings using state-of-the-art machine learning techniques built on features extracted from the audiovisual signal. Our data is targeted towards the identification of personality traits of students based on short recordings of experiments in a physics class. The project specific goals thus consist of 1) implementing feature extractors for audiovisual features and 2) evaluating the implemented software for automatic personality perception of students using the annotations of psychology researchers as a ground truth. The technology implemented in this project will be publicly available for further usage by other researchers from Digital Humanities.
Associated Software Products
Associated Research Areas
- Burkhardt, Paul Michael: Automatic prediction of students’ personality, self-concept and intelligence. On-going Bachelor thesis. 2016
A system for training end-to-end neural models to automatically recognize the motivational levels from students, using only their facial expressions as input, can be found here.