Computer Science City College of New York

CSC I6716 - Spring 2003

3D Computer Vision and Video Computing

Instructors

Instructor: Prof. Zhigang Zhu
Office: NAC 8/203A
Email: zhu@cs.ccny.cuny.edu

Class Time: Tuesday 1:50 - 4:20 pm
Room: NAC 4/125
Credits: 3.0
Office Hours: Tuesday 4:30 - 6:00 pm


Final reports due on May 27th, 2003  (Tuesday, before midnight).

The following only repeats what I have said in class in case some of you need a clarification.

(1) Each student OR each team needs to turn in both a soft copy and a hard copy of your project report. If you select to send in a single report and a single program from your team, you should clearly indicate who did which parts in your report, and each of you should have a summary and discussion section on what have been done and what need to be done fore the project. If you choose to send in a separate report on your own, you need to discuss the connection of your parts to your team's project. The grading will be 50% for the team collaboration and 50% for the individual contributions no matter what kinds of reports (team-based on individual-based) you are going to send.

(2) In your soft copy (via email), please send a zip file with your source code, any executables, description for running your program, and your project report (in either PDF or WORD).  Please write "Computer Vision" in your subject field when you send your email so that your soft copy will go to the right folder in my email-box.
(3) Your hard copy shall mainly include your project report (what, why, how and what's next), and probably printout of your code if you like. Please put your hard copies in my mailbox in the main office (NAC 8/206).

Course Update Information  (Bulletin Board )

May 13th, 2003. Agenda for final project presentations (Tuesday, May 20th, 1:50 - 5:10  pm, NAC 8/203)

May 9, 2003. Check out several stereo mosaics of CCNY campus!
                     From Camera Rotation:  Campus (1.15MB JPEG) , Grand Hall (514KB JPEG),  Grand Hall with Balloons (474KB JPEG)
                     From Camera Translation: Art Corridor Mosaic (400KB), ArtCorridor Panorama (800KB)

May 7, 2003.  Course Announcement: Professor Ioannis Stamos will offer 3D Photography (Tuesday 4:15-6:15pm) in the Graduate Center.

May 7, 2003.  More links to papers and web pages are provided in the Video Computing part.

April 28, 2003.  Scores for three homework assignments and the midterm

April 2nd, 2003. Please send me email on your choices of projects and teams. Two - three students may form a team, and select 1-2 topics for your project. Please send me your choice before Monday April 7 otherwise I will assign you topics in the class of April 8th.
April 1st, 2003,   Please check out Midterm Review
March 24, 2003. The Course Bulletin Board is up running. Students can discuss your questions over there.
March 18, 2003: Please check the revised course schedule for lectures, project topics,  midterm time.
March 12, 2003: Homework 3 online. Deadline: March 28 Friday before midnight - Extended to April 1 before class

Course Objectives

Computer vision has a rich history of work on stereo and visual motion, which has dealt with the problems of 3D reconstruction from binocular or N-ocular images, and structure from motion from video sequences. Recently, in addition to these traditional problems, the stereo and motion information present in multiple images or a video sequence is also being used to solve several other problems, for instance video mosaicing, video synthesis, video segmentation, video compression, video registration, and video surveillance an monitoring. This is summarized as Video Computing. Computer vision is playing an important and somewhat different role in solving these problems in video computing than the original image analysis considered in the early days of vision research.
 

Course Syllabus

The course "Video Computing and 3D Computer Vision" will include advanced topics in video computing as well as fundamentals in stereo and motion. The topics will be divided into three parts:

Part 1.  Computer  Vision Basics
  1. Introduction: Image, Vision and 3D Vision (ppt slides)  - Jan 28
  2. Sensors (ppt slides)  - Feb 4
  3. Image Formation and Processing (ppt slides,  matlab tutorial, Homework 1) -Feb 11 

**** Feb. 18, 2003, Snow Day: College Closed ****

  4. Features and Feature Extraction  (Part 1, Part 2, Homework 2) - Feb 25,  March 4

**** Feb 23. If you have not found the bug in my starting code for generating the Bayer pattern in homework 1, here is the corrected version. The assignments will be accepted by Feb. 28 Friday 5:00 pm without penalty  - please put your paper work in my mailbox in NAC 8/206.

Images could be used for your homework 

WaterLilies Image
Stereo Pair: Image 1, Image 2

Part 2. 3D Computer Vision
  5.  Camera Models (ppt slides) - March 4
  6.  Camera Calibration (ppt slides, Homework 3)  -March 11
  7.  Stereo Vision (Part 1, Part 2, Project Topics)  - March 18, March 25
  8.  Visual Motion (Part 1,  Part 2) - March 25, April 1
  8a. Project Discussions, Review for Exam   -April  1

   Midterm Exam - April 8th

**** April 15 Wed Schedule; April 16-24, 2003, Spring Recess ****

Part 3. Video Computing

  9.    Omnidirectional Stereo (Midterm Discussion; Reading: Shum-Kalai-Seitz , Zhu)- April 29
  10.  Video Mosaicing (Readings: Shum&Szeliski, Peleg,  Zhu)- May 6
  11.  Omnidirectional Cameras (Readings: Nayar,   Zhu)  - May 13
  12.  Applications - Readings:
        - Human Tracking (UCSD, UCF, Zhu-UMass, VSAM)
        - Image-Based Modeling and Rendering (Seitz-Dyer, UNC IBR, IBMR Workshop and More Links)
        - Content-Based Video Coding (Torr-Szeliski-Anandan, Zhu&Hanson, MPEG-4, MPEG 7 )

**** May 16 -23, 2003, Project Reports  ****

Textbook and References

Textbook:

    “Introductory Techniques for 3-D Computer Vision”,  Trucco and Verri, 1998.

References:

  1.     “Computer Vision – A Modern Approach” Forsyth and Ponce, 2003.
  2.     “Three Dimensional Computer Vision: A Geometric Viewpoint” O. Faugeras
  3.     “Image Processing, Analysis and Machine Vision” Sonika, Hlavac and Boyle, 1999
  4.     Kenneth R. Castleman, Digital Image Processing, Prentice Hall, Inc., 1996 (Chinese Translation by Zhigang Zhu, Xueyin Lin, Dingji Shi, Publishing House of Electronics Industry China, September 1998).  For those who are interested in a systematic description of image transforms (DFT, DCT, etc) and  image filtering. You may also find discussions on color and feature extraction.
  5. Jahne B, Digital Image Processing , Concept, Algorithms and Scientific Applications, Springer-Verlag, 1991- You may find a very good description of separable convolution kernels  and how to generate 1D/2D larger kernels from smaller 1D kernels in this book.
Supplements:  Online References and additional readings when necessary.
 

Grading and Prerequisites

The course will accommodate both graduate and senior undergraduate students with background in computer science, electrical and computer engineering, or applied mathematics. Students who take the course for credits will be required to finish 3 assignments of paperwork (30%), one midterm exam (20%), and  one programming project with exit interview (50%, including submit a report and give me a small presentation at the end of the semester). The topics of the projects will be given in the middle of the semester and will be related to the material presented in the lectures. Prereqs: Pre. CSc220 and Csc I0600.


Copyright @ Zhigang Zhu (email zhu@cs.ccny.cuny.edu ), Spring 2003.