Classifying Facial Action
Marian Stewart Bartlett, Paul A. Viola, Terrence J. Sejnowski, Beatrice A.
Golomb, Jan Larsen, Joseph C. Hager, and Paul Ekman
Advances in Neural Information Processing Systems 8,
D. Touretzky, M. Mozer, and M. Hasselmo (Eds.), MIT Press, Cambridge, MA,
1996. p. 823-829.
The Facial Action Coding System, (FACS), devised by Ekman and Friesen,
provides an objective means for measuring the facial muscle contractions
involved in a facial expression. In this paper, we approach automated
facial expression analysis by detecting and classifying facial actions. We
generated a database of over 1100 image sequences of 24 subjects performing
over 150 distinct facial actions or action combinations. We compare three
different approaches to classifying the facial actions in these images:
Holistic spatial analysis based on principal components of graylevel
images; explicit measurement of local image features such as wrinkles; and
template matching with motion flow fields. On a dataset containing six
individual actions and 20 subjects, these methods had 89%, 57%, and 85%
performances respectively for generalization to novel subjects. When
combined, performance improved to 92%.