In this paper, we explore the use of computer vision techniques to analyze students' moods during one-on-one teaching interactions. The eventual goal is to create automated tutoring systems that are sensitive to the student's mood and affective state. We find that the problem of accurately determining a child's mood from a single video frame is surprisingly difficult, even for humans. However when the system is allowed to make decisions based on information from 10 to 30 seconds of video, excellent performance may be obtained.
JF - 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011) PB - IEEE CY - Santa Barbara, CA SN - 978-1-4244-9140-7 ER - TY - CONF T1 - The computer expression recognition toolbox (CERT) T2 - 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011) Y1 - 2011 A1 - Littlewort, G. A1 - Whitehill, J. A1 - Wu, T. A1 - Fasel, I. A1 - Frank, M. A1 - Movellan, J. A1 - Bartlett, M. KW - 3D orientation KW - Accuracy KW - automatic real-time facial expression recognition KW - CERT KW - computer expression recognition toolbox KW - Detectors KW - dual core laptop KW - Emotion recognition KW - Encoding KW - extended Cohn-Kanade KW - Face KW - face recognition KW - facial action unit coding system KW - facial expression dataset KW - Facial features KW - FACS KW - Gold KW - Image coding KW - software tool KW - software tools KW - two-alternative forced choice task AB -We present the Computer Expression Recognition Toolbox (CERT), a software tool for fully automatic real-time facial expression recognition, and officially release it for free academic use. CERT can automatically code the intensity of 19 different facial actions from the Facial Action Unit Coding System (FACS) and 6 different prototypical facial expressions. It also estimates the locations of 10 facial features as well as the 3-D orientation (yaw, pitch, roll) of the head. On a database of posed facial expressions, Extended Cohn-Kanade (CK+[1]), CERT achieves an average recognition performance (probability of correctness on a two-alternative forced choice (2AFC) task between one positive and one negative example) of 90.1% when analyzing facial actions. On a spontaneous facial expression dataset, CERT achieves an accuracy of nearly 80%. In a standard dual core laptop, CERT can process 320 × 240 video images in real time at approximately 10 frames per second.
JF - 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011) PB - IEEE CY - Santa Barbara, CA SN - 978-1-4244-9140-7 ER - TY - CONF T1 - Learning to Make Facial Expressions T2 - IEEE 8th International Conference on Development and Learning, 2009. ICDL 2009 Y1 - 2009 A1 - Wu, T. A1 - Butko, N. A1 - Ruvulo, P. A1 - Bartlett, M. A1 - Movellan, J. KW - Actuators KW - Emotion recognition KW - face detection KW - face recognition KW - facial motor parameters KW - Feedback KW - Humans KW - learning (artificial intelligence) KW - Machine Learning KW - Magnetic heads KW - Pediatrics KW - real-time facial expression recognition KW - Robot sensing systems KW - robotic head KW - Robots KW - self-guided learning KW - Servomechanisms KW - Servomotors AB -This paper explores the process of self-guided learning of realistic facial expression production by a robotic head with 31 degrees of freedom. Facial motor parameters were learned using feedback from real-time facial expression recognition from video. The experiments show that the mapping of servos to expressions was learned in under one-hour of training time. We discuss how our work may help illuminate the computational study of how infants learn to make facial expressions.
JF - IEEE 8th International Conference on Development and Learning, 2009. ICDL 2009 PB - IEEE CY - Shanghai SN - 978-1-4244-4117-4 ER - TY - JOUR T1 - Toward Practical Smile Detection JF - IEEE Transactions on Pattern Analysis and Machine Intelligence Y1 - 2009 A1 - Whitehill, J. A1 - Littlewort, G. A1 - Fasel, I. A1 - Bartlett, M. A1 - Movellan, J. KW - Algorithms KW - Artificial intelligence KW - Automated KW - automatic facial expression recognition research KW - Biological Pattern Recognition KW - Biometry KW - Computer simulation KW - Computer vision KW - Computer-Assisted KW - Face KW - Face and gesture recognition KW - face recognition KW - feature representation KW - human-level expression recognition accuracy KW - illumination conditions KW - Image databases KW - Image Enhancement KW - Image Interpretation KW - image registration image representation KW - learning (artificial intelligence) KW - machine learning approaches KW - Machine Learning Models KW - n Humans KW - object detection KW - practical smile detection KW - Reproducibility of Results KW - Sensitivity and Specificity KW - Smiling KW - Subtraction Technique KW - training data set KW - visual databases AB -Machine learning approaches have produced some of the highest reported performances for facial expression recognition. However, to date, nearly all automatic facial expression recognition research has focused on optimizing performance on a few databases that were collected under controlled lighting conditions on a relatively small number of subjects. This paper explores whether current machine learning methods can be used to develop an expression recognition system that operates reliably in more realistic conditions. We explore the necessary characteristics of the training data set, image registration, feature representation, and machine learning algorithms. A new database, GENKI, is presented which contains pictures, photographed by the subjects themselves, from thousands of different people in many different real-world imaging conditions. Results suggest that human-level expression recognition accuracy in real-life illumination conditions is achievable with machine learning technology. However, the data sets currently used in the automatic expression recognition literature to evaluate progress may be overly constrained and could potentially lead research into locally optimal algorithmic solutions.
VL - 31 IS - 11 ER - TY - CONF T1 - A discriminative approach to frame-by-frame head pose tracking T2 - 8th IEEE International Conference on Automatic Face Gesture Recognition, 2008. FG '08 Y1 - 2008 A1 - Whitehill, J. A1 - Movellan, Javier R. KW - accuracy drift KW - continuous video sequence KW - controlled illumination condition KW - discriminative approach KW - face detection KW - face recognition KW - facial appearance KW - frame-by-frame head pose tracking KW - Humans KW - Image analysis KW - Image databases KW - Laboratories KW - Lighting KW - Magnetic heads KW - mean square error methods KW - pose estimation KW - Robustness KW - root-mean-square error tracking KW - System testing KW - Video sequences AB -We present a discriminative approach to frame-by-frame head pose tracking that is robust to a wide range of illuminations and facial appearances and that is inherently immune to accuracy drift. Most previous research on head pose tracking has been validated on test datasets spanning only a small (< 20) subjects under controlled illumination conditions on continuous video sequences. In contrast, the system presented in this paper was both trained and tested on a much larger database, GENKI, spanning tens of thousands of different subjects, illuminations, and geographical locations from images on the Web. Our pose estimator achieves accuracy of 5.82deg, 5.65deg, and 2.96deg root-mean-square (RMS) error for yaw, pitch, and roll, respectively. A set of 4000 images from this dataset, labeled for pose, was collected and released for use by the research community.
JF - 8th IEEE International Conference on Automatic Face Gesture Recognition, 2008. FG '08 PB - IEEE CY - Amsterdam SN - 978-1-4244-2153-4 ER -