CSc 59866/59867 Capstone I/II Fall 2012-Spring 2013
CS/CpE/EE Joint Senior Design Program (Planning Stage)
On Assistive Technology for Blind and Visually Impaired People

Instructors:
Professor Zhigang Zhu, Department of Computer Science
Professor Jizhong Xiao , Department of Electrical Engineering
City College of New York

Fall 2012 Class Meet Time & Location:    M,W 12:30-01:45PM  NAC-7/225
Fall 2012 Office Hours: Wednesdays 11:00 am - 12:00 pm;  2:00 pm - 3:00 pm, NAC 8/210  or 8/211

Fall 2012 (Final Grading) (academic calendar)

Week 1: August 27, 2012, Introduction (slides); Aug 29, 2012, Introduction to NSF-REM research projects

Week 2: September 05,2012, Reading Material Assignments and (Updated Reading List )
For each group of the papers, the first student shall do a 30 min presenation (with PPT slides), and the second student shall read the same paper and form questions for class discussion. Each class meet  we will have two presentations, each 30 minutes, in the order as listed in the reading list. Please send me the slides and questions before class.

Week 3: Sept 10, 2012 (Monday), Prof. Jizhong Xiao's lecture on Robotics Research and Education at CCNY and opportunities in joint projects; Sept 12, 2012: First student reading presentation section.

Week4: September 19, 2012 (Wed), Prof. YingLi Tian's lecture on computer vision technology for the blind

Week 6: October 3, 2012 (Wed), Lecture on camera models [pptx slides] [lecture notes] for preparing 3D computer vision related topics. Please go to Section 6.1 to work on the three questions. You will need to turn in your answers (in print) on Nov 5 in class.

Weeks 7-8: Please see the updated schedule for student presentations

Week 9: October 24, 2012 (Wed), PRISM talk by Dr. John Blaho on Bioinformatics and Systems Biology in Industrial Sponsored Research.

Week 10:
    October 29, 2012 (Mon), the last student presentation, and then we will start to discuss team assignments for project designs. Postponed to Nov 7 due to  Hurricane Sandy.
    October 31,  2012 (Wed), PRISM talk by Dr. Yaser Sheikh at CMU on  Exploring Visual Sensory Substitution with Wearable Cameras at the CMU Quality of Life Technology Center. (Cancelled due to Hurricane Sandy - will be rescheduled to the Spring Semester)

Week 11:
    November 05, 2012 (Mon), PRISM talk by Tao Wang at BAE Systems and CUNY Graduate Center on ADAPTIVE AND INTEGRATED MULTIMODAL SENSING AND PROCESSING
    November 07, 2012 (Wed), Student presentation slides are all up; we will also discuss team assignments for project designs. You may want to check out the project topic slides for discussions.

Weeks 12- 16 (five weeks).
    We are going to go over the various aspects of your proposed projects in the next five weeks.

    Tentative Five Week Schedule for you to consider
        Week 12: Overview of your proposed project. Please let me know if you want to go NCIIA E-TEAM or CCNY Kaylie Competition.
                        NCIIA E-TEAM Upcoming application deadlines: February 1, 2013; and May 10, 2013
                        Also watch out the Kaylie Deadline: early December?
        Week 13: Market needs and available technologies
        Week 14: Your proposed approaches and technical challenges
        Week 15: Budget for purchases and other expenses, and a possible business plan.
        Week 16: Finalize your proposed project
    Weekly plan:
        (1) Monday: group presentations (PPT slides, 15 minutes  each to the class);
        (2) Wednesday: group discussions in class and QAs with me;
        (3) After class group activities - including online search, proposal revision, and individual group discussion (at least 30 minute to make up the two classes in the Hurricane week).

    December 05, 2012 (Wed), PRISM Lecture Series Talk by Hao Tang at CUNY Graduate Center  and BMCC/CUNY on 3D SCENE MODELING AND UNDERSTANDING FROM IMAGE SEQUENCES and Smarting Sampling for Visually Impaired People

    The Last week: We are going to have the final proposal presentations on Monday (Dec 10), group discussions on Wednesday (Dec 12, with Wai, Hao, Joey and Alpha), and a summary and a 30-minutes test on Friday (Dec 14). The test will have some basic questions about your current understanding of your project (insights, reflections,  suggestions, questions, etc).

Background

Leveraging Prof. Zhigang Zhu’s expertise in computer vision and scene understanding, and Prof. John (Jizhong) Xiao’s expertise in robotic navigation, the two professors have expanded their research focus to developing a human centric assistive navigation system to help the blind people to achieve independent travel in unfamiliar environments. We have collaborations with the Computer Center for Visually Impaired People (CCVIP) at CUNY Baruch College, the NYS Commission for the Blind and Visually Handicapped (CBVH), and the New York Institute of Special Education. Their feedback indicate that many current assistive devices fall short of meeting the needs of blind users, and sometimes they even create more problems than they solve. Blind users demand compact, lightweight, inexpensive devices and heavily count on hearing and tactile to understand the scene. .

Description

With a team of faculty mentors (Professors Zhu and Xiao), entrepreneur advisers, blind consultants, graduate research assistants, and undergrad students, we have developed a cross-department joint senior design course for undergraduate seniors in CS, CpE and EE in developing multimodal, passive and unobtrusive techniques for helping visually impaired people to achieve independent travel in unfamiliar environments. We have obtained NCIIA funding support  and NSF funding support to carry out basic research and to run the cross-department joint senior design program for CS/CpE/EE seniors. The joint senior design program builds on our existing capstone design course structure in CS and EE departments, but with a new concentration on assistive technology for visually impaired people. 

In each year, the joint senior design course is a mandated two semester sequence for senior students in both CS and EE departments. In the first semester we will offer lectures on basic technologies in sensors, actuators, robotic navigation, vision algorithms, and assistive technologies by the instructors. The general lectures will introduce important aspects of a business plan such as project management, intellectual property (IP), entrepreneurship. In addition, the lecture series on entrepreneurship at GSoE will be utilized where professionals are invited to share their real world experiences.

In the second part of the first semester, under the guidance of the instructors, the undergrad seniors will form teams to survey the state-of-the-art technologies in the three challenges areas (i.e., multimodal sensing approaches, data interpretation algorithms, and display methods), perform patent searches, conduct marketing analyses, and write project proposals which shall include design ideas, a reasonable budget, a management plan with milestones, and a business plan. CS/CpE/EE students are encouraged to form multidisciplinary teams and work collaboratively to contribute their different expertise in the projects. The faculty mentors will review the proposals and give senior design students feedback to refine  their projects
In the second semester, the student teams are expected to implement design ideas, prototype, test, and evaluate different designs. Students will also have the opportunities to perform usability study on vision impaired users in collaboration with NYS CBVH therefore better understanding their needs to improve the designs and to create more appropriate business plans.

The prominent teams with innovative ideas/technology and good business plans will be recommended to compete for the Kaylie Prize for Entrepreneurship. The winning team will be supported to continue the effort in summer.  

Useful Links and Update Information

NSF EFRI-M3C Funding Support for Research (For more information, please see a news post about the project.)

NSF GARDE Funding Support for Course Development

NCIIA funding support [planning] [regular]

Our 2011-2012 Capstone Team V.I.S.T.A. together with two other students in our labs won the $50000 Kaylie Prize (See also The City College Kaylie Prize for Entrepreneurship ) A video demonstrating the early prototype of the device can be found here.

Professor Zhigang Zhu's Visual Computing Laboratory

Professor John (Jizhong) Xiao's Robotics Laboratory

Computer Center for Visually Impaired People (CCVIP) at CUNY Baruch College

NYS Commission for the Blind and Visually Handicapped (CBVH)

New York Institute for Special Education (NYISE)


Copyright @ Zhigang Zhu,  2012-2013