Page Registration

CSc Senior Capstone Sequence 2004-2005
Computer Science - The City College of New York

Vision, Video and Virtual Reality

Instructor: Professor Zhigang Zhu

Digital Classroom: Registration and Integration of Powerpoint Slides, Video Images and Whiteboard Pages

The commonly used classroom and seminar presentation tools are PowerPoint©(PPT) slides, overhead projections, and blackboards /whiteboards. In order to digitize the classroom handwriting contents to create e-lectures, we use a low-cost Mimio© digital whiteboards system [1] to substitute for the use of the blackboard. Therefore, the "sensors" we are using in our portable presentation system for a classroom setting are a PPT slide capturer, a Mimio Virtual Ink handwriting system, and a digital video camera (to capture classroom video/audio) [2,3,4]. All the sensors are controlled by the laptop used for the presentation and can be easily managed by the instructor/lecturer using the system.

The PPT slide presentation and the whiteboard system when used together could provide a better means to present a lecture in a classroom. When the PPT slides are projected on the whiteboard, where the instructor annotates, modifies or expands the PPT presentation, we need to geometrically register the images from the two sources to create high quality electronic presentations. We have developed algorithms to fulfill this goal [4]. Here is the overview of our approach. First, the corresponding PPT slides, handwriting pages, and video frames are matched up via the synchronization mechanism in the following manner. When the instructor changes slides, a new slide image S is saved by the PPT slide capturer and a slide-only video frame V1 is extracted (after a delay of few frames to ensure a stable frame). When a new handwriting page H is saved by the Mimio Virtual Ink, the timing information will locate this page with the right slide S, and the video frame V2 with both slide projection and handwriting contents will be saved automatically. Second, the PPT slide S and the handwriting page H are registered by way of the video frames V1 and V2. The slide-video registration is implemented by finding the boundary of the projection area in the video frame V1. The video-whiteboard registration is fulfilled by matching the handwriting contents saved in the Mimio page H and the video frames (V1 and V2).

Requirements:
However we still need to improve the robustness and speed of the algorithms. For example, we need to find better way to register a PPT slide with its video image. We also need to develop a more robust algorithm to find the initial match between a video frame and a handwriting page. The basic requirements are

(1) Please first read the paper listed in [4] and then find out problems that you need to solve.
(2) Develop algorithms to solve the problems and generate the desired output
(3) Write a technical report

Tools:
C++, Matlab

Input:
Four images: A PPT slide image, a corresponding handwriting page image, two video frames (with and without handwriting). The distortions in video frames have been removed.

Output:

A integrated image with both high resolution PPT and handwriting contents.

Reading:
[1] Mimio Virtual Ink, http://www.mimio.com/index.shtml
[2]. Virtual Classroom - the Next Generation e-Learning System
[3]. Z. Zhu, C. McKittrick and W. Li, Virtualized Classroom – automated production, media integration and user-customized presentation, The 4th International Workshop on Multimedia Data and Document Engineering (with CVPR’04), July 2, 2004, Washington DC
[4]. W. Li, H. Tang and Z. Zhu, Automated registration of high-resolution images from slide presentations and whiteboard handwritings via a low-cost digital video camera. The 2nd IEEE Workshop on Image and Video Registration (with CVPR’04), July 2, 2004, Washington DC (accepted)