We present the Computer Expression Recognition Toolbox (CERT), a software tool for fully automatic real-time facial expression recognition, and officially release it for free academic use. CERT can automatically code the intensity of 19 different facial actions from the Facial Action Unit Coding System (FACS) and 6 different prototypical facial expressions. It also estimates the locations of 10 facial features as well as the 3-D orientation (yaw, pitch, roll) of the head. On a database of posed facial expressions, Extended Cohn-Kanade (CK+[1]), CERT achieves an average recognition performance (probability of correctness on a two-alternative forced choice (2AFC) task between one positive and one negative example) of 90.1% when analyzing facial actions. On a spontaneous facial expression dataset, CERT achieves an accuracy of nearly 80%. In a standard dual core laptop, CERT can process 320 × 240 video images in real time at approximately 10 frames per second.
JF - 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011) PB - IEEE CY - Santa Barbara, CA SN - 978-1-4244-9140-7 ER - TY - JOUR T1 - Toward Practical Smile Detection JF - IEEE Transactions on Pattern Analysis and Machine Intelligence Y1 - 2009 A1 - Whitehill, J. A1 - Littlewort, G. A1 - Fasel, I. A1 - Bartlett, M. A1 - Movellan, J. KW - Algorithms KW - Artificial intelligence KW - Automated KW - automatic facial expression recognition research KW - Biological Pattern Recognition KW - Biometry KW - Computer simulation KW - Computer vision KW - Computer-Assisted KW - Face KW - Face and gesture recognition KW - face recognition KW - feature representation KW - human-level expression recognition accuracy KW - illumination conditions KW - Image databases KW - Image Enhancement KW - Image Interpretation KW - image registration image representation KW - learning (artificial intelligence) KW - machine learning approaches KW - Machine Learning Models KW - n Humans KW - object detection KW - practical smile detection KW - Reproducibility of Results KW - Sensitivity and Specificity KW - Smiling KW - Subtraction Technique KW - training data set KW - visual databases AB -Machine learning approaches have produced some of the highest reported performances for facial expression recognition. However, to date, nearly all automatic facial expression recognition research has focused on optimizing performance on a few databases that were collected under controlled lighting conditions on a relatively small number of subjects. This paper explores whether current machine learning methods can be used to develop an expression recognition system that operates reliably in more realistic conditions. We explore the necessary characteristics of the training data set, image registration, feature representation, and machine learning algorithms. A new database, GENKI, is presented which contains pictures, photographed by the subjects themselves, from thousands of different people in many different real-world imaging conditions. Results suggest that human-level expression recognition accuracy in real-life illumination conditions is achievable with machine learning technology. However, the data sets currently used in the automatic expression recognition literature to evaluate progress may be overly constrained and could potentially lead research into locally optimal algorithmic solutions.
VL - 31 IS - 11 ER - TY - CONF T1 - Auditory mood detection for social and educational robots T2 - IEEE International Conference on Robotics and Automation, 2008. ICRA 2008 Y1 - 2008 A1 - Ruvolo, P. A1 - Fasel, I. A1 - Movellan, J. KW - auditory mood detection KW - Computer vision KW - educational robot KW - Educational robots KW - Emotion recognition KW - emotional speech database KW - face detection KW - hearing KW - interactive robotic application KW - learning (artificial intelligence) KW - Machine Learning KW - Mood Prototypes KW - object recognition KW - Robotics and Automation Robots KW - social mood KW - social robot KW - Speech KW - USA Councils AB -Social robots face the fundamental challenge of detecting and adapting their behavior to the current social mood. For example, robots that assist teachers in early education must choose different behaviors depending on whether the children are crying, laughing, sleeping, or singing songs. Interactive robotic applications require perceptual algorithms that both run in real time and are adaptable to the challenging conditions of daily life. This paper explores a novel approach to auditory mood detection which was born out of our experience immersing social robots in classroom environments. We propose a new set of low-level spectral contrast features that extends a class of features which have proven very successful for object recognition in the modern computer vision literature. Features are selected and combined using machine learning approaches so as to make decisions about the ongoing auditory mood. We demonstrate excellent performance on two standard emotional speech databases (the Berlin Emotional Speech [W. Burkhardt et al., 2005], and the ORATOR dataset [H. Quast, 2001]). In addition we establish strong baseline performance for mood detection on a database collected from a social robot immersed in a classroom of 18-24 months old children [J. Movellan er al., 2007]. This approach operates in real time at little computational cost. It has the potential to greatly enhance the effectiveness of social robots in daily life environments.
JF - IEEE International Conference on Robotics and Automation, 2008. ICRA 2008 PB - IEEE CY - Pasadena, CA SN - 978-1-4244-1646-2 ER - TY - THES T1 - Learning Real-Time Object Detectors: Probabilistic Generative Approaches T2 - Department of Cognitive Science Y1 - 2006 A1 - Fasel, I. JF - Department of Cognitive Science PB - University of California, San Diego CY - San Diego VL - Doctoral dissertation ER - TY - COMP T1 - MPT: the Machine Perception Toolbox Y1 - 2005 A1 - Fasel, I. A1 - Fortenberry, B. A1 - Movellan, J. ER - TY - CONF T1 - Analysis of machine learning methods for real-time recognition of facial expressions from video T2 - Computer Vision and Pattern Recognition: Face Processing Workshop Y1 - 2004 A1 - Littlewort, G. A1 - Bartlett, M. A1 - Fasel, I. A1 - Chenu, J. A1 - Movellan, J. JF - Computer Vision and Pattern Recognition: Face Processing Workshop ER - TY - JOUR T1 - Towards social robots: Automatic evaluation of human-robot interaction by face detection and expression classification JF - Advances in Neural Information Processing Systems Y1 - 2004 A1 - Littlewort, G. A1 - Bartlett, M. A1 - Chenu, J. A1 - Fasel, I. A1 - Kanda, T. A1 - Ishiguro, H. A1 - Movellan, J. VL - 16 ER -