Abstract

We present progress on a system for fully automatic measurement of facial expressions. The system pairs automatic face detection with texture-based expression classification using machine learning techniques. These machine learning techniques can be applied to code any facial expression dimension given a set of labeled training images. We present results for automatic recognition of facial action codes, as well as recognition of full expressions of basic emotion. The present system attains 93% accuracy for person-independent recognition of expressions of basic emotion in a 7-alternative forced choice (neutral, anger, disgust, fear, joy, sadness, surprise) on the Cohn-Kanade AU-Coded Facial Expression Database. The system also recognizes 18 action units of the Facial Action Coding System whether they occur individually or in combination. Mean agreement was 94.5% with human FACS codes in the Cohn-Kanade database. All results are for generalization to novel subjects in a fully automatic system. The outputs of the detectors are continuous, and provide information about expression intensity at each video frame. This opens up the possibility of studying dynamics at time scales previously intractible by human coding due to time considerations. Preliminary results for applying this system to spontaneous facial behavior will be presented.