PanoMovie: Dynamic Panoramas from Multiple Cameras


Back to Zhigang's Homepage | Computer Science | School of Engineering | CCNY

Interfaces using mosaic representations have become common for a single camera that is rotating or translating. However, there has been little work in combined mosaics from many cameras with highly varying viewpoints, and across dynamic video streams. A collective representation of a set of sensor suites across many (mobile) platforms in some fused manner remains unexplored, and is a challenging and complex problem.  As an example, think of  building a surrounding 2D mosaic from the multiple overlapping static snapshots from different viewpoints. This representation can be characterized as a seamless mosaic with multiple-centers-of-projection (MCOP). Previous work uses a single camera moving on a planned path to generate MCOP images for image-based rendering. The MCOP concept can be extended to multiple cameras widely distributed in space. The possibilities become even more exciting when we consider a dynamic mosaic interface of overlapping video streams of many distributed cameras for applications such as surveillance and maintenance..

At UMass, we have already demonstrated this concept with two stationary cameras to produce a wider field of view of a moving object (Fig. 1 ).  The video  demonstrates the train moving seamlessly through the two overlapping views by tying adjacent video streams together. This approach allows us to create an integrated representation across sensors from many platforms.


(1)  (2) 
(3) 
 Figure 1. Dynamic mosaic from multiple video streams. (1) and (2) are left and right frames, and (3) is the corresponding fused frames. Better perception of the train can be obtained in the fused image. Click the corresponding images to view the movies (in AVI)

Collaborators:

Edward M. Riseman, Professor
Howard Schultz, Senior Research Scientist
Andrew Fagg, Senior Research Scientist
Chris Denio, system programmer


Related projects:

 Kollmorgen Corp. (industrial contract), High-Resolution Panoramic Sensing Project, 1/11/99-6/30/99 , PI (Schultz), Co-PIs (Riseman, Zhu)