In this paper, we explore the use of computer vision techniques to analyze students' moods during one-on-one teaching interactions. The eventual goal is to create automated tutoring systems that are sensitive to the student's mood and affective state. We find that the problem of accurately determining a child's mood from a single video frame is surprisingly difficult, even for humans. However when the system is allowed to make decisions based on information from 10 to 30 seconds of video, excellent performance may be obtained.
10 aautomated facial affect analysis 10 aautomated tutoring system 10 abehavioural sciences computing 10 acomputer vision technique 10 aContext 10 adecision making 10 aeducation 10 aEmotion recognition 10 aface recognition 10 aHuman 10 ahuman computer interaction 10 aLabeling 10 aMachine Learning 10 aMood 10 an Histograms 10 aone-on-one tutoring application 10 as Intelligent tutoring systems 10 astudent mood analysis 1 aButko, N. 1 aTheocharous, G. 1 aPhilipose, M. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/automated-facial-affect-analysis-one-one-tutoring-applications 02299nas a2200445 4500008004100000020002200041245005500063210004900118260003700167520099600204653001901200653001301219653005401232653000901286653004401295653001401339653002101353653002401374653001301398653002501411653000901436653002101445653003701466653003001503653002001533653000901553653000901562653001701571653001801588653001901606653003901625100001901664700001801683700001101701700001401712700001401726700001701740700001701757856007901774 2011 eng d a978-1-4244-9140-7 00 aThe computer expression recognition toolbox (CERT) 0 acomputer expression recognition toolbox CERT aSanta Barbara, CA bIEEE c03/2011 3 aWe present the Computer Expression Recognition Toolbox (CERT), a software tool for fully automatic real-time facial expression recognition, and officially release it for free academic use. CERT can automatically code the intensity of 19 different facial actions from the Facial Action Unit Coding System (FACS) and 6 different prototypical facial expressions. It also estimates the locations of 10 facial features as well as the 3-D orientation (yaw, pitch, roll) of the head. On a database of posed facial expressions, Extended Cohn-Kanade (CK+[1]), CERT achieves an average recognition performance (probability of correctness on a two-alternative forced choice (2AFC) task between one positive and one negative example) of 90.1% when analyzing facial actions. On a spontaneous facial expression dataset, CERT achieves an accuracy of nearly 80%. In a standard dual core laptop, CERT can process 320 × 240 video images in real time at approximately 10 frames per second.
10 a3D orientation 10 aAccuracy 10 aautomatic real-time facial expression recognition 10 aCERT 10 acomputer expression recognition toolbox 10 aDetectors 10 adual core laptop 10 aEmotion recognition 10 aEncoding 10 aextended Cohn-Kanade 10 aFace 10 aface recognition 10 afacial action unit coding system 10 afacial expression dataset 10 aFacial features 10 aFACS 10 aGold 10 aImage coding 10 asoftware tool 10 asoftware tools 10 atwo-alternative forced choice task 1 aLittlewort, G. 1 aWhitehill, J. 1 aWu, T. 1 aFasel, I. 1 aFrank, M. 1 aMovellan, J. 1 aBartlett, M. uhttps://rubi.ucsd.edu/content/computer-expression-recognition-toolbox-cert 01688nas a2200385 4500008004100000020002200041245004000063210004000103260002800143520060700171653001400778653002400792653001900816653002100835653002800856653001300884653001100897653003900908653002100947653001900968653001500987653004401002653002601046653001701072653001101089653002501100653002001125653001601145100001101161700001401172700001501186700001701201700001701218856006701235 2009 eng d a978-1-4244-4117-4 00 aLearning to Make Facial Expressions 0 aLearning to Make Facial Expressions aShanghai bIEEE c06/2009 3 aThis paper explores the process of self-guided learning of realistic facial expression production by a robotic head with 31 degrees of freedom. Facial motor parameters were learned using feedback from real-time facial expression recognition from video. The experiments show that the mapping of servos to expressions was learned in under one-hour of training time. We discuss how our work may help illuminate the computational study of how infants learn to make facial expressions.
10 aActuators 10 aEmotion recognition 10 aface detection 10 aface recognition 10 afacial motor parameters 10 aFeedback 10 aHumans 10 alearning (artificial intelligence) 10 aMachine Learning 10 aMagnetic heads 10 aPediatrics 10 areal-time facial expression recognition 10 aRobot sensing systems 10 arobotic head 10 aRobots 10 aself-guided learning 10 aServomechanisms 10 aServomotors 1 aWu, T. 1 aButko, N. 1 aRuvulo, P. 1 aBartlett, M. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/learning-make-facial-expressions 02714nas a2200361 4500008004100000020002200041245006200063210006200125260003200187520158600219653002801805653002001833653002201853653002301875653002401898653003001922653001901952653001201971653003601983653003902019653002102058653002002079653002302099653003502122653001602157653001702173653001102190653001702201100001502218700001402233700001702247856008802264 2008 eng d a978-1-4244-1646-2 00 aAuditory mood detection for social and educational robots 0 aAuditory mood detection for social and educational robots aPasadena, CA bIEEE c05/2008 3 aSocial robots face the fundamental challenge of detecting and adapting their behavior to the current social mood. For example, robots that assist teachers in early education must choose different behaviors depending on whether the children are crying, laughing, sleeping, or singing songs. Interactive robotic applications require perceptual algorithms that both run in real time and are adaptable to the challenging conditions of daily life. This paper explores a novel approach to auditory mood detection which was born out of our experience immersing social robots in classroom environments. We propose a new set of low-level spectral contrast features that extends a class of features which have proven very successful for object recognition in the modern computer vision literature. Features are selected and combined using machine learning approaches so as to make decisions about the ongoing auditory mood. We demonstrate excellent performance on two standard emotional speech databases (the Berlin Emotional Speech [W. Burkhardt et al., 2005], and the ORATOR dataset [H. Quast, 2001]). In addition we establish strong baseline performance for mood detection on a database collected from a social robot immersed in a classroom of 18-24 months old children [J. Movellan er al., 2007]. This approach operates in real time at little computational cost. It has the potential to greatly enhance the effectiveness of social robots in daily life environments.
10 aauditory mood detection 10 aComputer vision 10 aeducational robot 10 aEducational robots 10 aEmotion recognition 10 aemotional speech database 10 aface detection 10 ahearing 10 ainteractive robotic application 10 alearning (artificial intelligence) 10 aMachine Learning 10 aMood Prototypes 10 aobject recognition 10 aRobotics and Automation Robots 10 asocial mood 10 asocial robot 10 aSpeech 10 aUSA Councils 1 aRuvolo, P. 1 aFasel, I. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/auditory-mood-detection-social-and-educational-robots 01620nas a2200349 4500008004100000020002200041245006600063210006600129260003200195520052200227653001900749653001900768653002800787653003500815653001300850653003900863653001400902653002300916653002400939653001700963653001100980653003900991653002101030653000901051653002501060653001501085653001501100653003001115100001501145700001701160856009301177 2008 eng d a978-1-4244-2661-4 00 aAutomatic cry detection in early childhood education settings 0 aAutomatic cry detection in early childhood education settings aMonterey, CA bIEEE c08/2008 3 aWe present results on applying a novel machine learning approach for learning auditory moods in natural environments [1] to the problem of detecting crying episodes in preschool classrooms. The resulting system achieved levels of performance approaching that of human coders and also significantly outperformed previous approaches to this problem [2].
10 aAcoustic noise 10 aauditory moods 10 aautomatic cry detection 10 abehavioural sciences computing 10 aDeafness 10 aearly childhood education settings 10 aeducation 10 aEducational robots 10 aEmotion recognition 10 ahuman coders 10 aHumans 10 alearning (artificial intelligence) 10 aMachine Learning 10 aMood 10 apreschool classrooms 10 aPrototypes 10 aRobustness 10 aWorking environment noise 1 aRuvolo, P. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/automatic-cry-detection-early-childhood-education-settings