Abstract
This paper demonstrates a new visual motion estimation technique that is able to recover high degree-of-freedom articulated human body configurations in complex video sequences. We introduce the use and integration of a mathematical technique, the product of exponential maps and twist motions, into a differential motion estimation. This results in solving simple linear systems, and enables us to recover robustly the kinematic degrees-of-freedom in noise and complex self occluded configurations. A new factorization technique lets us also recover the kinematic chain model itself. We are able to track several human walk cycles, several wallaby hop cycles, and two walk cycels of the famous movements of Eadweard Muybridge's motion studies from the last century. To the best of our knowledge, this is the first computer vision based system that is able to process such challenging footage.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ayer, S. and Sawhney, H.S. 1995. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and mdl encoding. In Int. Conf. Computer Vision, Cambridge, MA, pp. 777–784.
Basu, S., Essa, I.A., and Pentland, A.P. 1996. Motion regularization for model-based head tracking. In International Conference on Pattern Recognition.
Bergen, J.R., Anandan, P., Hanna, K.J., and Hingorani, R. 1992. Hierarchical model-based motion estimation. In ECCV, pp. 237– 252.
Black, M.J. and Anandan, P. 1996. The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields. Computer Vision and Image Understanding, 63(1):75–104.
Black, M.J. and Yacoob, Y. 1995. Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion. In ICCV.
Black, M.J., Yacoob, Y., Jepson, A.D., and Fleet, D.J. 1997. Learning parameterized models of image motion. In CVPR.
Blake, A., Isard, M., and Reynard, D. 1995. Learning to track the visual motion of contours. J. Artificial Intelligence.
Bregler, C. and Malik, J. 1998. Estimating and tracking kinematic chains. In IEEE Conf. On Computer Vision and Pattern Recognition.
Clergue, E., Goldber, M., Madrane, N., and Merialdo, B. 1995. Automatic face and gestual recognition for video indexing. In Proc. of the Int.Workshop on Automatic Face-and Gesture-Recognition, Zurich, 1995.
Concalves, L., Bernardo, E.D., Ursella, E., and Perona, P. 1995. Monocular tracking of the human arm in 3d. In Proc. Int. Conf. Computer Vision.
Davis, J.W. and Bobick, A.F. 1997. The representation and recognition of human movement using temporal templates. In CVPR.
Dempster, A.P., Laird, N.M., and Rubin, B.D. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39.
Gavrila, D.M. and Davis, L.S. 1950. Towards 3-d model-based tracking and recognition of human movement: A multi-view approach. In Proc. Of the Int. Workshop on Automatic Face-and Gesture-Recognition, Zurich.
Hogg, D. 1983. A program to see a walking person. Image Vision Computing, 5(20).
Jepson, A. and Black, M.J. 1993. Mixture models for optical flow computation. In Proc. IEEE Conf. Computer Vision Plattern Recognition, New York, pp. 760–761.
Ju, S.X., Black, M.J., and Yacoob, Y. 1996. Cardboard people: A parameterized model of articulated motion. In 2nd Int. Conf. On Automatic Face-and Gesture-Recognition, Killington, Vermon, pp. 38–44.
Kakadiaris, I.A. and Metaxas, D. 1996. Model-based estimation of 3d human motion with occlusion based on active multiviewpoint selection. In CVPR.
Lucas, B.D. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. 7th Int. Joinnt Conf. on Art. Intell.
Murray, M.P., Drought, A.B., and Kory, R.C. 1964.Walking patterns of normal men. Journal of Bone and Joint Surgery, 46-A(2):335–360.
Murray, R.M., Li, Z., and Sastry, S.S. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press.
Muybridge, E. 1901. The Human Figure in Motion. Various Publishers, latest edition by Dover Publications.
Pentland, A. and Horowitz, B. 1991. Recovery of nonrigid motion and structure. IEEE Transactions on PAMI, 13(7):730–742.
Regh, J.M. and Kanade, T. 1995. Model-based tracking of selfoccluding articulated objects. In Proc. Int. Conf. Computer Vision.
Rohr, K. 1993. Incremental recognition of pedestrians from image sequences. In Proc. IEEE Comput. Soc. Conf. Comput. Vision and Pattern Recogn. New York City, pp. 8–13.
Shi, J. and Tomasi, C. 1994. Good features to tract. In CVPR.
Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography:Afactorization method. Int. J. of Computer Vision, 9(2):137–154.
Weiss, Y. and Adelson, H.E. 1995. Perceptually organized EM: A framework for motion segmentation that combines information about form and motion. Technical Report 315, M.I.T Media Lab.
Weiss, Y. and Adelson, H.E. 1996. A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. In Proc. IEEE Conf. Computer Vision Pattern Recognition.
Wren, C., Azarbayejani, A., Darrell, T., and Pentland, A. 1995. Pfinder: Real-time tracking of the human body. In SPIE Conference on Integration Issues in Large Commercial Media Delivery Systems, vol. 2615.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Bregler, C., Malik, J. & Pullen, K. Twist Based Acquisition and Tracking of Animal and Human Kinematics. International Journal of Computer Vision 56, 179–194 (2004). https://doi.org/10.1023/B:VISI.0000011203.00237.9b
Issue Date:
DOI: https://doi.org/10.1023/B:VISI.0000011203.00237.9b