In this paper, we explore the use of computer vision techniques to analyze students' moods during one-on-one teaching interactions. The eventual goal is to create automated tutoring systems that are sensitive to the student's mood and affective state. We find that the problem of accurately determining a child's mood from a single video frame is surprisingly difficult, even for humans. However when the system is allowed to make decisions based on information from 10 to 30 seconds of video, excellent performance may be obtained.
JF - 2011 IEEE International Conference on Automatic Face Gesture Recognition and Workshops (FG 2011) PB - IEEE CY - Santa Barbara, CA SN - 978-1-4244-9140-7 ER - TY - JOUR T1 - Infomax Control of Eye Movements JF - IEEE Transactions on Autonomous Mental Development Y1 - 2010 A1 - Butko, N. A1 - Movellan, J. KW - active information gathering KW - autonomous computer program KW - autonomous physical agent KW - Computer vision KW - dynamic tracking task KW - Eye movement KW - eye movement strategy KW - face detection KW - faces KW - Infomax control KW - motor system KW - object detection KW - optimal control KW - optimal eye movement controller KW - policy gradient KW - probabilistic model KW - sensory system KW - static scenes KW - Visual Perception KW - visual search KW - visual system AB -Recently, infomax methods of optimal control have begun to reshape how we think about active information gathering. We show how such methods can be used to formulate the problem of choosing where to look. We show how an optimal eye movement controller can be learned from subjective experiences of information gathering, and we explore in simulation properties of the optimal controller. This controller outperforms other eye movement strategies proposed in the literature. The learned eye movement strategies are tailored to the specific visual system of the learner-we show that agents with different kinds of eyes should follow different eye movement strategies. Then we use these insights to build an autonomous computer program that follows this approach and learns to search for faces in images faster than current state-of-the-art techniques. The context of these results is search in static scenes, but the approach extends easily, and gives further efficiency gains, to dynamic tracking tasks. A limitation of infomax methods is that they require probabilistic models of uncertainty of the sensory system, the motor system, and the external world. In the final section of this paper, we propose future avenues of research by which autonomous physical agents may use developmental experience to subjectively characterize the uncertainties they face.
VL - 2 IS - 2 ER - TY - CONF T1 - Learning to Make Facial Expressions T2 - IEEE 8th International Conference on Development and Learning, 2009. ICDL 2009 Y1 - 2009 A1 - Wu, T. A1 - Butko, N. A1 - Ruvulo, P. A1 - Bartlett, M. A1 - Movellan, J. KW - Actuators KW - Emotion recognition KW - face detection KW - face recognition KW - facial motor parameters KW - Feedback KW - Humans KW - learning (artificial intelligence) KW - Machine Learning KW - Magnetic heads KW - Pediatrics KW - real-time facial expression recognition KW - Robot sensing systems KW - robotic head KW - Robots KW - self-guided learning KW - Servomechanisms KW - Servomotors AB -This paper explores the process of self-guided learning of realistic facial expression production by a robotic head with 31 degrees of freedom. Facial motor parameters were learned using feedback from real-time facial expression recognition from video. The experiments show that the mapping of servos to expressions was learned in under one-hour of training time. We discuss how our work may help illuminate the computational study of how infants learn to make facial expressions.
JF - IEEE 8th International Conference on Development and Learning, 2009. ICDL 2009 PB - IEEE CY - Shanghai SN - 978-1-4244-4117-4 ER - TY - CONF T1 - Visual saliency model for robot cameras T2 - IEEE International Conference on Robotics and Automation, 2008. ICRA 2008 Y1 - 2008 A1 - Butko, N. A1 - Zhang, L. A1 - Cottrell, G. A1 - Movellan, J. KW - Application software KW - approximation theory KW - Bayes methods KW - Bayesian methods KW - Bayesian model KW - camera control KW - Cameras KW - Central Processing Unit KW - Computational efficiency KW - Computational modeling KW - Explosions KW - fast approximation KW - human visual attention KW - Humans KW - Open loop systems KW - robot cameras KW - robot vision KW - Robot vision systems KW - robotic application KW - task free conditions KW - visual saliency model AB -Recent years have seen an explosion of research on the computational modeling of human visual attention in task free conditions, i.e., given an image predict where humans are likely to look. This area of research could potentially provide general purpose mechanisms for robots to orient their cameras. One difficulty is that most current models of visual saliency are computationally very expensive and not suited to real time implementations needed for robotic applications. Here we propose a fast approximation to a Bayesian model of visual saliency recently proposed in the literature. The approximation can run in real time on current computers at very little computational cost, leaving plenty of CPU cycles for other tasks. We empirically evaluate the saliency model in the domain of controlling saccades of a camera in social robotics situations. The goal was to orient a camera as quickly as possible toward human faces. We found that this simple general purpose saliency model doubled the success rate of the camera: it captured images of people 70% of the time, when compared to a 35% success rate when the camera was controlled using an open-loop scheme. After 3 saccades (camera movements), the robot was 96% likely to capture at least one person. The results suggest that visual saliency models may provide a useful front end for camera control in robotics applications.
JF - IEEE International Conference on Robotics and Automation, 2008. ICRA 2008 PB - IEEE CY - Pasadena, CA SN - 978-1-4244-1646-2 ER - TY - CONF T1 - Learning to Learn T2 - IEEE International Conference on Development and Learning Y1 - 2007 A1 - Butko, N. A1 - Movellan, J. JF - IEEE International Conference on Development and Learning ER -