CCNY Seminar in Computer Vision, Perceptual Robotics and Video Computing

---------------------------------------------------------------------------------------------------

Motion Layers Extraction in the Presence of Occlusion Using Graph Cuts

Dr. Mubarak Shah
Computer Vision Lab, School of Computer Science
University of Central Florida, Orlando, FL 32816

http://www.cs.ucf.edu/~vision/

Date: October 11, 2004

Time: 1:30 pm – 2:30 pm

Location: NAC 8/207, City College of New York

Host: Prof. Zhigang Zhu <zzhu@ccny.cuny.edu>

Extracting layers from video is very important for video representation, analysis, compression, and synthesis. Assuming that a scene can be approximately described by multiple planar regions, we have developed a robust and novel approach to automatically extract a set of affine or projective transformations induced by these regions, correctly detect the occlusion pixels over multiple consecutive frames, and accurately segment the scene into several motion layers. First, using correspondences in two frames a number of seed regions are determined, and the regions are expanded and the outliers are rejected employing the two-terminal graph cuts method integrated with the level sets representation. Second, these initial regions are merged into several initial layers according to motion similarity. Third, the occlusion order constraints on multiple frames are explored, which guarantee that the occlusion area increases with the temporal order in a short period and effectively maintains segmentation consistency over multiple consecutive frames. Then, the correct layer segmentation is obtained by using the alpha expansion algorithm for solving the multiple labeling problem, and the occlusions between the overlapping layers are explicitly determined.

In this talk several experimental results will be demonstrated to show that our approach is effective and robust. Furthermore, based on motion segmentation, three novel applications: layer-based compression, layer-based video registration and layer- based video completion will be demonstrated.

For more details visit:

http://www.cs.ucf.edu/~vision/projects/motion_layer_extraction/

Dr. Mubarak Shah, a professor of Computer Science, and the founding director of the Computer Vision Laboratory at University of Central Florida (UCF), is a researcher in computer vision. He is a co-author of two books Video Registration (2003) and Motion-Based Recognition (1997), both by Kluwer Academic Publishers. He has supervised several Ph.D., M.S., and B.S. students to completion, and is currently directing twenty Ph.D. and several B.S. students. He has published close to one hundred fifty papers in leading journals and conferences on topics including activity and gesture recognition, violence detection, event ontology, object tracking (fixed camera, moving camera, multiple overlapping and non-overlapping cameras), video segmentation, story and scene segmentation, view morphing, ATR, wide-baseline matching, and video registration. . Dr. Shah is a fellow of IEEE, was an IEEE Distinguished Visitor speaker for 1997-2000, and is often invited to present seminars, tutorials and invited talks all over the world. He received the Harris Corporation Engineering Achievement Award in 1999, the TOKTEN awards from UNDP in 1995, 1997, and 2000; Teaching Incentive Program award in 1995 and 2003, Research Incentive Award in 2003, and IEEE Outstanding Engineering Educator Award in 1997. He is an editor of international book series on “Video Computing”; editor in chief of Machine Vision and Applications journal, and an associate editor Pattern Recognition journal. He was an associate editor of the IEEE Transactions on PAMI, and a guest editor of the special issue of International Journal of Computer Vision on Video Computing.