On Environment Modeling for Visual Navigation

- Imaging, Modeling and Representation of Real Scenes

Table of Contents (TOC in Chinese )

1.1  Problems
1.2  Background

1.2.1  The Marr Paradigm
1.2.2  The DARPA Visual Navigation Efforts
1.2.3 Obstacle Detection Techniques
1.2.4 Omnidirectional Vision and Panoramic Vision

1.3  Our Approach: Multi-Scale and Full View Vision

1.3.1 Overview
1.3.2 Contributions
1.3.3 Organizations

2.1  Introduction
2.2      Motion Filtering and Image Stabilization

2.2.1 Vehicular Motion Model (Appendix 2.1, Appendix 2.2)
2.2.2 Image Rectification
2.2.3 Motion Filtering Algorithms
2.2.4  PVI and EPI generation: Examples

2.3   Panoramic EPI Analysis Approach

2.3.1 Motion Texture and Motion Occlusion Models (Appendix 2.3)
2.3.2 GFOD: Large Gaussian-Windowed Fourier Orientation Detector (Appendix 2.4)
2.3.3 Depth Belief Map and Data Selection
2.3.4 Motion Boundary Localization and Depth Interpolation

2.4  Panoramic Modeling and Generalized Landmark Selection

2.4.1 Image Rectification and Stabilization
2.4.2 Panoramic Depth Acquisition: Parallel Processing
2.4.3 Fusion of Depth and Intensity Maps
2.4.4 Generalized Landmark Selection

2.5  Summary and Discussions

3.1  Introduction
3.2  Omnidirectional Vision-Based Eigenspace Representation

3.2.1. Problem Statement
3.2.2 Eigenspace Representation
3.2.3 Omnidirectional Eigenspace Representation

3.3   Real-Time Omnidirectional Imaging Sensor

3.3.1 Sensor Geometry (Appendix 3.1)
3.3.2 Practical Imaging System and System Calibration
3.3.4. Ground Projection and Image Rectification
3.3.5 Ground Feature Analysis in Polar Coordinate System (Appendix 3.2)

3.4 Rotational-Invariant Feature Space of Omnidirectional Images

3.4.1  Radial Principle Component Analysis (PCA)
3.4.2  Orientation Fourier Transform
3.4.3  Orientation Estimation

3.5  OVINN: Omnidirectional Vision-based Neural Networks

3.5.1 Problem Statement
3.5.2  The Model of the ROVINN
3.5.3  Implementation Issues

3.6 Experimental Results and Analysis

3.6.1 System Architecture
3.6.2  Data Collection
3.6.3  Training and Testing

3.7  Summary and Discussions

4.1 Introduction
4.2  Principle of Planar Gaze Transformation

4.2.1 Theory (Appendix 4.1)
4.2.2 Properties (Appendix 4.2)
4.2.3 Applications

4.3  Binocular Vision System  Using a Single Camera

4.3.1  Design I: the Left-Right Partition (How to make it compact)
4.3.2  Design II: the Up_Down Partition (How to have a wide FOV)

4.4 Real-Time Obstacle Detection Algorithm

4.4.1  Goal and Assumptions
4.4.2  Statistical Modeling (Appendix 4.3)
4.4.3 The Basic Algorithm
4.4.4 Implementation and Performance (Appendix 4.4)

4.5  Dynamic Gaze Transformation

4.5.1 Iterative Approach
4.5.2 Image Stabilization Method
4.5.3  Generalized HOUGH Transform Approach

4.6  Summary and Discussion

5.1 Scene Modeling and Interconnection: a Systems Approach
5.2  Sensing Integration: a Compact Full-View Visual Sensor
5.3  Data Integration and Interconnection among Sub-Systems
5.4  Human-Robot Interaction and Vision Enhancements
5.5  Conclusions and Discussions

1.  Z. Zhu, Full View Spatio-Temporal Visual Navigation - Imaging, Modeling and Representation of Real Scenes, China Higher Education Press, December 2001, First Hundred National Excellent Doctorate Dissertations Series ( in Chinese)  Abstract,  TOC
2.  Z. Zhu and A. R. Hanson, 3D LAMP:  a New Layered Panoramic Representation, The Eighth IEEE  International Conference on Computer Vision, Vancouver, Canada, July 2001, vol II, 723-730.
3.  Zhigang Zhu, Guangyou Xu, Xueyin Lin, Panoramic EPI Generation and Analysis of Video from a Moving Platform with Vibration, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 23-25 June, 1999, Fort Collins, Colorado, vol 2, pp 531-537. (cvpr99.ps.gz: 2.8 MB) ( MPEG Demo: 3.8 M)
4. Zhigang Zhu, Guangyou Xu, Bo Yang, Dingji Shi, Xueyin Lin, VISATRAM: A real-time vision system for automatic traffic monitoring, Image and Vision Computing Journal,  18(10), July 2000: pp 781-794. (jivc99.ps.gz: 893KB)
5.   Zhigang Zhu, Shiqiang Yang, Guangyou Xu, Xueyin Lin, Dingji Shi, Fast Road Classification and Orientation Estimation Using Omni-View Images and Neural Networks, IEEE Transactions on Image Processing, vol 7, no 8, August 1998: pp 1182-1197 
6.   Zhigang Zhu, Guangyou Xu, Yudong Yang, Jesse S. Jin, Camera stabilization based on 2.5D motion estimation and inertial motion filtering, IEEE Internaltional Conference on Intelligent Vehicles, Oct 28-30, 1998, Stuttgart, Germany. (iv98.ps.gz: 244 KB)  ( MPEG Demo: 1.3 M)
7.   Zhigang Zhu, Xueyin Lin, Dingji Shi, Guangyou Xu, A Single Camera Stereo System for Obstacle Detection, World Multiconference on Systemics, Cybernetics and Informatics (SCI'98) /4th International Conference on Information Systems Analysis and Synthesis (ISAS'98), vol 3, July 12-16, 1998, Orlando, U.S.A, pp 230-237  (sci98.ps.gz: 840 KB)
8.   Zhigang Zhu, Shiqiang Yang, Dingji Shi, Guangyou Xu, Better road following by integrating omni-view images and neural nets, 1998 World Congress on Computational Intelligence, Proceedings of IJCNN,  vol 2, May 4-9, 1998, Anchorage, Alaska, USA, pp 974-979 (wcci98.ps.gz: 764 KB)
9. Zhigang Zhu, Guangyou Xu, Dingji Shi, Qualitative estimations of range and motion using spatio-temporal textural images, Proc. 12th International Conference on Pattern Recognition, Jerusalem, Israel, Oct. 9-13, 1994: pp 736-738
10. Zhigang Zhu, Guangyou Xu, Shaoyun Chen, Xueyin Lin, Dynamic obstacle detection through cooperation of purposive visual modules of color stereo and motion, Proc. IEEE International Conference on Robotics and Automation, pp 1916-1922, San Diego, May 1994
11. Xueyin Lin, Zhigang Zhu, Detecting Height From Constrained Motion, Proc. IEEE 3rd International Conference on Computer Vision, pp 503-506, Osaka, Japan, 1990
12. Zhigang Zhu, Xueyin Lin,  Real-time algorithms for obstacle avoidance by using reprojection transformation, Proc. IAPR Workshop on Machine Vision and Applications, Tokyo, Japan, 1990

Back to Zhigang's Homepage | Computer Science | School of Engineering | CCNY