Rate-invariant comparisons of covariance paths for visual speech recognition

Jingyong Su, Anuj Srivastava, Fillipe Souza, Sudeep Sarkar

Research output: Contribution to conferencePaperpeer-review

Abstract

An important problem in speech, and generally activity, recognition is to develop analyses that are invariant to the execution rates. We introduce a theoretical framework that provides a parametrization-invariant metric for comparing parametrized paths on Riemannian manifolds. Treating instances of activities as parametrized paths on a Riemannian manifold of covariance matrices, we apply this framework to the problem of visual speech recognition from image sequences. We represent each sequence as a path on the space of covariance matrices, each covariance matrix capturing spatial variability of visual features in a frame, and perform simultaneous pairwise temporal alignment and comparison of paths. This removes the temporal variability and helps provide a robust metric for visual speech classification. We evaluated this idea on the OuluVS database and the rank-1 nearest neighbor classification rate improves from 32% to 57% due to temporal alignment.

Original languageEnglish
DOIs
StatePublished - 2013
Event2013 4th National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, NCVPRIPG 2013 - Jodhpur, Rajasthan, India
Duration: Dec 18 2013Dec 21 2013

Conference

Conference2013 4th National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, NCVPRIPG 2013
Country/TerritoryIndia
CityJodhpur, Rajasthan
Period12/18/1312/21/13

Fingerprint

Dive into the research topics of 'Rate-invariant comparisons of covariance paths for visual speech recognition'. Together they form a unique fingerprint.

Cite this