On Environment Modeling for Visual Navigation

- Imaging, Modeling and Representation of Real Scenes

Zhigang Zhu

Ph.D. Thesis
Department of Computer Science and Technology
Tsinghua University, Beijing

Note: The documents in English are not the exact translations of the original thesis, and are only partially available. Most parts of these documents are recently updated, but some materials in the original version may not included in the English version. A complete and revised version of the thesis has be published by China Higher Education Press in December 2001 (in Chinese).

Table of Contents (TOC in Chinese )

Abstract (PDF)

Acknowledgments (Abstract and Acknowledgments in Chinese)

Chapter 1. Problems and Solutions (PDF, html)
1.1  Problems
1.2  Background
1.2.1  The Marr Paradigm
1.2.2  The DARPA Visual Navigation Efforts
1.2.3 Obstacle Detection Techniques
1.2.4 Omnidirectional Vision and Panoramic Vision
1.3  Our Approach: Multi-Scale and Full View Vision
1.3.1 Overview
1.3.2 Contributions
1.3.3 Organizations
Chapter 2. Panoramic Vision for Landmark Recognition (PDF, html, Appendixes)
2.1  Introduction
2.2      Motion Filtering and Image Stabilization
2.2.1 Vehicular Motion Model (Appendix 2.1, Appendix 2.2)
2.2.2 Image Rectification
2.2.3 Motion Filtering Algorithms
2.2.4  PVI and EPI generation: Examples
2.3   Panoramic EPI Analysis Approach
2.3.1 Motion Texture and Motion Occlusion Models (Appendix 2.3)
2.3.2 GFOD: Large Gaussian-Windowed Fourier Orientation Detector (Appendix 2.4)
2.3.3 Depth Belief Map and Data Selection
2.3.4 Motion Boundary Localization and Depth Interpolation
2.4  Panoramic Modeling and Generalized Landmark Selection
2.4.1 Image Rectification and Stabilization
2.4.2 Panoramic Depth Acquisition: Parallel Processing
2.4.3 Fusion of Depth and Intensity Maps
2.4.4 Generalized Landmark Selection
2.5  Summary and Discussions
Chapter 3. Omnidirectional Vision for Road Understanding (PDF [5])
3.1  Introduction
3.2  Omnidirectional Vision-Based Eigenspace Representation
3.2.1. Problem Statement
3.2.2 Eigenspace Representation
3.2.3 Omnidirectional Eigenspace Representation
3.3   Real-Time Omnidirectional Imaging Sensor
3.3.1 Sensor Geometry (Appendix 3.1)
3.3.2 Practical Imaging System and System Calibration
3.3.4. Ground Projection and Image Rectification
3.3.5 Ground Feature Analysis in Polar Coordinate System (Appendix 3.2)
3.4 Rotational-Invariant Feature Space of Omnidirectional Images
3.4.1  Radial Principle Component Analysis (PCA)
3.4.2  Orientation Fourier Transform
3.4.3  Orientation Estimation
3.5  OVINN: Omnidirectional Vision-based Neural Networks
3.5.1 Problem Statement
3.5.2  The Model of the ROVINN
3.5.3  Implementation Issues
3.6 Experimental Results and Analysis
3.6.1 System Architecture
3.6.2  Data Collection
3.6.3  Training and Testing
3.7  Summary and Discussions
Chapter 4. Stereo Vision for Obstacle Detection (PDF)
4.1 Introduction
4.2  Principle of Planar Gaze Transformation
4.2.1 Theory (Appendix 4.1)
4.2.2 Properties (Appendix 4.2)
4.2.3 Applications
4.3  Binocular Vision System  Using a Single Camera
4.3.1  Design I: the Left-Right Partition (How to make it compact)
4.3.2  Design II: the Up_Down Partition (How to have a wide FOV)
4.4 Real-Time Obstacle Detection Algorithm
4.4.1  Goal and Assumptions
4.4.2  Statistical Modeling (Appendix 4.3)
4.4.3 The Basic Algorithm
4.4.4 Implementation and Performance (Appendix 4.4)
4.5  Dynamic Gaze Transformation
4.5.1 Iterative Approach
4.5.2 Image Stabilization Method
4.5.3  Generalized HOUGH Transform Approach
4.6  Summary and Discussion

Chapter 5. POST: a Multi-Scale and Full-View Vision Approach
5.1 Scene Modeling and Interconnection: a Systems Approach
5.2  Sensing Integration: a Compact Full-View Visual Sensor
5.3  Data Integration and Interconnection among Sub-Systems
5.4  Human-Robot Interaction and Vision Enhancements
5.5  Conclusions and Discussions
Chapter 6.  Conclusions and Future Directions

References (PDF)

Some Related Publications
  1.  Z. Zhu, Full View Spatio-Temporal Visual Navigation - Imaging, Modeling and Representation of Real Scenes, China Higher Education Press, December 2001, First Hundred National Excellent Doctorate Dissertations Series ( in Chinese)  Abstract,  TOC
  2.  Z. Zhu and A. R. Hanson, 3D LAMP:  a New Layered Panoramic Representation, The Eighth IEEE  International Conference on Computer Vision, Vancouver, Canada, July 2001, vol II, 723-730.
  3.  Zhigang Zhu, Guangyou Xu, Xueyin Lin, Panoramic EPI Generation and Analysis of Video from a Moving Platform with Vibration, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 23-25 June, 1999, Fort Collins, Colorado, vol 2, pp 531-537. (cvpr99.ps.gz: 2.8 MB) ( MPEG Demo: 3.8 M)
  4. Zhigang Zhu, Guangyou Xu, Bo Yang, Dingji Shi, Xueyin Lin, VISATRAM: A real-time vision system for automatic traffic monitoring, Image and Vision Computing Journal,  18(10), July 2000: pp 781-794. (jivc99.ps.gz: 893KB)
  5.   Zhigang Zhu, Shiqiang Yang, Guangyou Xu, Xueyin Lin, Dingji Shi, Fast Road Classification and Orientation Estimation Using Omni-View Images and Neural Networks, IEEE Transactions on Image Processing, vol 7, no 8, August 1998: pp 1182-1197 
  6.   Zhigang Zhu, Guangyou Xu, Yudong Yang, Jesse S. Jin, Camera stabilization based on 2.5D motion estimation and inertial motion filtering, IEEE Internaltional Conference on Intelligent Vehicles, Oct 28-30, 1998, Stuttgart, Germany. (iv98.ps.gz: 244 KB)  ( MPEG Demo: 1.3 M)
  7.   Zhigang Zhu, Xueyin Lin, Dingji Shi, Guangyou Xu, A Single Camera Stereo System for Obstacle Detection, World Multiconference on Systemics, Cybernetics and Informatics (SCI'98) /4th International Conference on Information Systems Analysis and Synthesis (ISAS'98), vol 3, July 12-16, 1998, Orlando, U.S.A, pp 230-237  (sci98.ps.gz: 840 KB)
  8.   Zhigang Zhu, Shiqiang Yang, Dingji Shi, Guangyou Xu, Better road following by integrating omni-view images and neural nets, 1998 World Congress on Computational Intelligence, Proceedings of IJCNN,  vol 2, May 4-9, 1998, Anchorage, Alaska, USA, pp 974-979 (wcci98.ps.gz: 764 KB)
  9. Zhigang Zhu, Guangyou Xu, Dingji Shi, Qualitative estimations of range and motion using spatio-temporal textural images, Proc. 12th International Conference on Pattern Recognition, Jerusalem, Israel, Oct. 9-13, 1994: pp 736-738
  10. Zhigang Zhu, Guangyou Xu, Shaoyun Chen, Xueyin Lin, Dynamic obstacle detection through cooperation of purposive visual modules of color stereo and motion, Proc. IEEE International Conference on Robotics and Automation, pp 1916-1922, San Diego, May 1994
  11. Xueyin Lin, Zhigang Zhu, Detecting Height From Constrained Motion, Proc. IEEE 3rd International Conference on Computer Vision, pp 503-506, Osaka, Japan, 1990
  12. Zhigang Zhu, Xueyin Lin,  Real-time algorithms for obstacle avoidance by using reprojection transformation, Proc. IAPR Workshop on Machine Vision and Applications, Tokyo, Japan, 1990

Back to Zhigang's Homepage | Computer Science | School of Engineering | CCNY