CB3M: Content-Based 3D Mosaics with Dynamic Pushbroom Stereo
Back to Zhigang's Homepage | Computer Science | Grove School of Engineering | CCNY
We propose a content-based three-dimensional (3D) mosaic representation  for long video sequences of 3D and dynamic scenes captured by a camera on a mobile platform. The motion of the camera has a dominant direction of motion (as on an airplane or ground vehicle), but 6 degrees-of-freedom (DOF) motion is allowed. In the first step, a pair of generalized parallel-perspective (pushbroom) stereo mosaics is generated that captured both the 3D and dynamic aspects of the scene under the camera coverage. In the second step, a segmentation-based stereo matching algorithm is applied to extract parametric representation of the color, structure and motion of the dynamic and/or 3D objects in urban scenes where a lot of planar surfaces exist. Based on these results, the content-based 3D mosaic (CB3M) representation is created, which is a highly compressed visual representation for very long video sequences of dynamic 3D scenes. Our goal is to acquire geo-referenced 3D mosaic maps with motion tracking information for 3D (moving) targets using a light aerial vehicle flying over or a ground vehicle move across an unknown 3D environment.

Related Publications

  1. H. Tang, Z. Zhu, G. Wolberg and J. R. Layne, Dynamic 3D Urban Scene Modeling Using Multiple Pushbroom Mosaics, the Third International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT 2006), University of North Carolina, Chapel Hill, USA, June 14-16, 2006, accepted.
  2. Z. Zhu, H. Tang, Content-Based Dynamic 3D Mosaics, IEEE Workshop on Three-Dimensional Cinematography (3DCINE'06),  June 22, New York City (in conjunction with CVPR), accepted
  3. Z. Zhu, H. Tang, G. Wolberg and J. R. Layne, Content-Based 3D Mosaics for Dynamic Urban 3D Scenes. SPIE Defense and Security Symposium 2006, 17 - 21 April 2006, Orlando, Florida, USA
  4. Z. Zhu, Y.-C. Hu, Gamma/X-Ray Linear Pushbroom Stereo for 3D Cargo Inspection. SPIE Defense and Security Symposium 2006, 17 - 21 April 2006, Orlando, Florida, USA
  5. Z. Zhu, H. Tang, G. Wolberg and J. R. Layne, Content-Based 3D Mosaic Representation for Video of Dynamic 3D Scenes. IEEE/AIPR Workshop 2005: Multi-Modal Imaging, Washington DC, October 19-21, 2005
  6. Z. Zhu,  H. Tang, B. Shen, G. Wolberg, 3D and Moving Target Extraction from Dynamic Pushbroom Stereo Mosaics, IEEE Workshop on Advanced 3D Imaging for Safety and Security, June 25, 2005, San Diego, CA, USA
  7. Z. Zhu, A. R. Hanson, Mosaic-Based 3D Scene Representation and Rendering, Special Session on Interactive Representation of Still and Dynamic Scenes, the Eleventh International Conference on Image   Processing, Genova, Italy, September  11-14,  2005,.
  8. Z. Zhu, L. Zhao, J. Lei, 3D Measurements in Cargo Inspection with a Gamma-Ray Linear Pushbroom Stereo System, IEEE Workshop on Advanced 3D Imaging for Safety and Security, June 25, 2005, San Diego, CA, USA
  9. Z. Zhu, E. M. Riseman, A. R. Hanson and H. Schultz, An Efficient Method for Geo-Referenced Video Mosaicing for Environmental Monitoring.  Machine Vision Applications Journal,  16(4), 2005, 203-126
  10. Z. Zhu, E. M. Riseman, A. R. Hanson, Generalized Parallel-Perspective Stereo Mosaics from Airborne Videos, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 2, Feb 2004, pp 226-237.

Collaborators:

Students:

Related Grants