CSc Senior Capstone Sequence 2004-2005
Computer Science - The City College of New York

Vision, Video and Virtual Reality

Instructor: Zhigang Zhu
Associate Professor
Computer Science and Computer Engineering
The City College of New York and Graduate Center
The City University of New York (CUNY)

Fall 2004 Class Meet @ 12:00 pm - 2:00 pm (M,W) at NAC 8/203A
Spring 2005 Class Meet @ 9:30 pm - 12:00 pm (Th) at NAC 8/203A

The Capstone course will last for two semesters. The first semester will be a study on a number of basic principles of relative technologies in Computing Vision, Video Computing and Virtualized Reality. The topics will include:

Introduction: Vision, Video and Virtual Reality (3 lectures)

Lecture 1: Introduction - Objectives, Lectures and Project Topics - Aug 30 (Mon)
Lecture 2: Project Topic 1: Augmented New York City (Aug-NYC) - Sep 1 (Wed)
Lecture 3: Project Topic 2: Virtualized Classroom (VC) - Sep 8 (Wed)

Visual Sensors, Image and Video Formation (3 lectures)

Lecture 4: Visual Sensors - Sep 13 (Mon)
Lecture 5: Image and Video Formations - Sep 20 (Mon)
Lecture 6: Omnidirectional Imaging - Sep 22 (Wed)

Low-Level Vision: Image Enhancement, Edges and Contours (3 lectures)

Lecture 7: Image Enhancement - Sep 27 (Mon)
Lecture 8: Edge Detection - Sep 29 (Wed)
Lecture 9: Image Segmentation - Oct 4 (Mon)

Camera Models and Virutal Environments (4 lectures)

Lecture 10: Motion Layer Extraction Using Graph Cut (Dr. Mubarak Shah) - NAC 8 / 207, Oct 11 (Mon, 1:30 pm - 2:30 pm)
Lecture 11: Student Presentations: Virutal Environments in Actions - Oct 13 (Wed)
Lecture 12: 3D Computer Vision: Camera Models - Oct 18 (Mon)
Lecture 13: 3D Computer Vision: Camera Calibration - Oct 20 (Wed)
Lab: Projects and Tools (OpenGL, Java3D) - Oct 25 (Mon) - no class meet

3D Computer Vision (Stereo and Motion): From 3D to 2D (4 lectures)

Lecture 14: Stereo Vision (I) - Oct 27 (Wed)
Lecture 15: Stereo Vision (II) -Nov 1 (Mon)
Lecture 16: Visual Motion (I) - Nov 3 (Wed)
Lecture 17: Visual Motion (II) - Nov 8 (Mon)

Visual Representation: Video Mosaics, Layered Representations (3 lectures)

Lecture 18: Omnidrectional Stereo - Nov 10 (Wed)
Lab: OpenGL and Java3D - Nov 15 (Mon), Nov 17 (Wed), no class meet
Lecture 19: Video Mosaics and Layered Representations - Nov 22 (Mon)
Lecture 20: OpenGL with C++ and Qt (by Molina Edgardo) - Nov 29 (Mon)

Image-Based Rendering and Content-Based Video Computing (4 lectures)

Lecture 21: Arts, Designs, Organization and Documentation (by Robert Hill)- Dec 1 (Wed)
Lecture 22: Content-Based A/V and MPEG 4/7 in C++ and Java (by Emmanuel Velasco) - Dec 6(Mon)
Lecture 23: OpenGL with Java 3D (by Peter Giang) - Dec 8 (Wed)
Lecture 24: Active Objects and Hyperlinks in VEs (by Krishnan Ram)- Dec 13 (Mon)
Lecture 25: ACM Multimedia Conference 2004 Reviews (by Weihong Li) - Dec 14 (Tue)

The lectures will be combined with the discussions of Capstone projects for real world applications. The possible project topics might be among the following:

1. Augmented New York City: Vision,Video and Virtual Reality in Traffic and Surveillance
2. Virtualized Classroom: Registration and Integration of Powerpoint Slides, Video Images and Whiteboard Pages
3. Magic Glass – A Special Magnifying Glass that Sees Different Attributes of a Color Image

These topics integrate computer vision, video processing and virtual reality applications, and are quite open for further research and/or development. The projects are subject to changes based on the first semester course progress. Students are going to work in teams, each of them with about 5 students working on the same project. In the second semester, the project teams formed in the first semester will mainly focus on the implementations of the projects. Basically, you can select any programming languages you would like to use for your projects, but you need to aware that you must be familiar with the appropriate programming languages used for a selected project.

Textbook and References

Textbook:

“Introductory Techniques for 3-D Computer Vision”, Trucco and Verri, 1998.

References:

OpenGL:
    NeHe OpenGL Tutorials (with code samples) - recommended for this course
    Code samples for the OpenGL v1.1 Programming Guide (Redbook)
    Nate Robins - OpenGL - Tutors