In this paper, we explore the use of computer vision techniques to analyze students' moods during one-on-one teaching interactions. The eventual goal is to create automated tutoring systems that are sensitive to the student's mood and affective state. We find that the problem of accurately determining a child's mood from a single video frame is surprisingly difficult, even for humans. However when the system is allowed to make decisions based on information from 10 to 30 seconds of video, excellent performance may be obtained.
10 aautomated facial affect analysis 10 aautomated tutoring system 10 abehavioural sciences computing 10 acomputer vision technique 10 aContext 10 adecision making 10 aeducation 10 aEmotion recognition 10 aface recognition 10 aHuman 10 ahuman computer interaction 10 aLabeling 10 aMachine Learning 10 aMood 10 an Histograms 10 aone-on-one tutoring application 10 as Intelligent tutoring systems 10 astudent mood analysis 1 aButko, N. 1 aTheocharous, G. 1 aPhilipose, M. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/automated-facial-affect-analysis-one-one-tutoring-applications 02633nas a2200397 4500008004100000022001400041245003700055210003700092300001100129490000600140520152800146653003301674653003201707653003001739653002001769653002601789653001701815653002601832653001901858653001001877653002001887653001701907653002101924653002001945653003601965653002002001653002402021653001902045653001802064653002202082653001802104653001802122100001402140700001702154856006402171 2010 eng d a1943-0604 00 aInfomax Control of Eye Movements 0 aInfomax Control of Eye Movements a91-107 0 v2 3 aRecently, infomax methods of optimal control have begun to reshape how we think about active information gathering. We show how such methods can be used to formulate the problem of choosing where to look. We show how an optimal eye movement controller can be learned from subjective experiences of information gathering, and we explore in simulation properties of the optimal controller. This controller outperforms other eye movement strategies proposed in the literature. The learned eye movement strategies are tailored to the specific visual system of the learner-we show that agents with different kinds of eyes should follow different eye movement strategies. Then we use these insights to build an autonomous computer program that follows this approach and learns to search for faces in images faster than current state-of-the-art techniques. The context of these results is search in static scenes, but the approach extends easily, and gives further efficiency gains, to dynamic tracking tasks. A limitation of infomax methods is that they require probabilistic models of uncertainty of the sensory system, the motor system, and the external world. In the final section of this paper, we propose future avenues of research by which autonomous physical agents may use developmental experience to subjectively characterize the uncertainties they face.
10 aactive information gathering 10 aautonomous computer program 10 aautonomous physical agent 10 aComputer vision 10 adynamic tracking task 10 aEye movement 10 aeye movement strategy 10 aface detection 10 afaces 10 aInfomax control 10 amotor system 10 aobject detection 10 aoptimal control 10 aoptimal eye movement controller 10 apolicy gradient 10 aprobabilistic model 10 asensory system 10 astatic scenes 10 aVisual Perception 10 avisual search 10 avisual system 1 aButko, N. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/infomax-control-eye-movements 01688nas a2200385 4500008004100000020002200041245004000063210004000103260002800143520060700171653001400778653002400792653001900816653002100835653002800856653001300884653001100897653003900908653002100947653001900968653001500987653004401002653002601046653001701072653001101089653002501100653002001125653001601145100001101161700001401172700001501186700001701201700001701218856006701235 2009 eng d a978-1-4244-4117-4 00 aLearning to Make Facial Expressions 0 aLearning to Make Facial Expressions aShanghai bIEEE c06/2009 3 aThis paper explores the process of self-guided learning of realistic facial expression production by a robotic head with 31 degrees of freedom. Facial motor parameters were learned using feedback from real-time facial expression recognition from video. The experiments show that the mapping of servos to expressions was learned in under one-hour of training time. We discuss how our work may help illuminate the computational study of how infants learn to make facial expressions.
10 aActuators 10 aEmotion recognition 10 aface detection 10 aface recognition 10 afacial motor parameters 10 aFeedback 10 aHumans 10 alearning (artificial intelligence) 10 aMachine Learning 10 aMagnetic heads 10 aPediatrics 10 areal-time facial expression recognition 10 aRobot sensing systems 10 arobotic head 10 aRobots 10 aself-guided learning 10 aServomechanisms 10 aServomotors 1 aWu, T. 1 aButko, N. 1 aRuvulo, P. 1 aBartlett, M. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/learning-make-facial-expressions 02735nas a2200409 4500008004100000020002200041245004400063210004400107260003200151520155400183653002501737653002501762653001801787653002101805653001901826653001901845653001201864653002801876653002901904653002701933653001501960653002301975653002701998653001102025653002202036653001802058653001702076653002502093653002402118653002502142653002602167100001402193700001402207700001702221700001702238856007002255 2008 eng d a978-1-4244-1646-2 00 aVisual saliency model for robot cameras 0 aVisual saliency model for robot cameras aPasadena, CA bIEEE c05/2008 3 aRecent years have seen an explosion of research on the computational modeling of human visual attention in task free conditions, i.e., given an image predict where humans are likely to look. This area of research could potentially provide general purpose mechanisms for robots to orient their cameras. One difficulty is that most current models of visual saliency are computationally very expensive and not suited to real time implementations needed for robotic applications. Here we propose a fast approximation to a Bayesian model of visual saliency recently proposed in the literature. The approximation can run in real time on current computers at very little computational cost, leaving plenty of CPU cycles for other tasks. We empirically evaluate the saliency model in the domain of controlling saccades of a camera in social robotics situations. The goal was to orient a camera as quickly as possible toward human faces. We found that this simple general purpose saliency model doubled the success rate of the camera: it captured images of people 70% of the time, when compared to a 35% success rate when the camera was controlled using an open-loop scheme. After 3 saccades (camera movements), the robot was 96% likely to capture at least one person. The results suggest that visual saliency models may provide a useful front end for camera control in robotics applications.
10 aApplication software 10 aapproximation theory 10 aBayes methods 10 aBayesian methods 10 aBayesian model 10 acamera control 10 aCameras 10 aCentral Processing Unit 10 aComputational efficiency 10 aComputational modeling 10 aExplosions 10 afast approximation 10 ahuman visual attention 10 aHumans 10 aOpen loop systems 10 arobot cameras 10 arobot vision 10 aRobot vision systems 10 arobotic application 10 atask free conditions 10 avisual saliency model 1 aButko, N. 1 aZhang, L. 1 aCottrell, G. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/visual-saliency-model-robot-cameras 00263nas a2200097 4500008004100000245002200041210002200063100001400085700001700099856004900116 2007 eng d 00 aLearning to Learn 0 aLearning to Learn 1 aButko, N. 1 aMovellan, J. uhttps://rubi.ucsd.edu/content/learning-learn