Computer Vision and Image Processing – Spring 2022
- Instructor: Professor Zhigang Zhu
- The CUNY Graduate Center and City College
- Course Codes: CSC 74030-01 (58079) Computer Vision and Image Processing
- Time: Wednesday 9:30 – 11:30 am; Location (Hybrid): (1) In-person at The CUNY Graduate Center Rm 6496 (noted in the syllabus), (2) Online [Class Zoom Link]
- Office Hours: Thursday 2:00 pm – 4:00 pm [Office Hours Zoom Link].
Course Update Information
- 02/02/2022. The first day of this class.
- 02/02/2022. If you want to start to think about it earlier, here are some Project Ideas and Requirements. The user name and password will be announced in class.
- 03/08/2022. Grading for Assignment 1 and IC CAE bonus points.
- 03/22/2022. Grading for Assignments 1-2 and IC CAE bonus points.
- 04/25/2022. Grading for Assignments 1-3, Exam, Project Proposals and IC CAE bonus points. We will discuss the Exam on 04/27 in class.
- 04/26/2022. Grading for Exam Updated.
- 05/11/2022. Grading for Project Presentations and Everything Else Before That. Please submit your report to firstname.lastname@example.org by the end of May 16, 2022.
- 05/24/2022. Final Grading will be submitted on Thursday 05/26/2022 on or before noon. Have a great summer!
This course will cover the fundamental work on color, shapes, stereo and visual motion, which has dealt with the problems of image understanding, 3D reconstruction from multiple images, and structure from motion with video sequences. In addition to these traditional problems, we will also showcase a recent example in successfully using image processing, computer vision and machine learning techniques for transforming large transportation centers into Smart and Accessible Transportation Hubs (SAT-Hubs) for serving people with disabilities. Moreover, the best successful vision system that computer vision researchers can learn from is the human vision system. Therefore this course will briefly discuss human vision science and explore how the brain sees the world too.
Course Syllabus and Tentative Schedule (mm/dd)
Part I. Introduction and Human Vision
- I-1. Human and Machine Vision (slides); Human Eyes (slides) (video) – 02/02 (Assignment 1 on I-1, I-3 and I-4)
- I-2. Visual Brain (slides) (video) -02/09
- I-3. Depth (slides) (video for depth and color)-02/16
- I-4. Color (slides) -02/16
Part II. 3D Computer Vision
- II-1. Camera Models (slides) (video) (lecture notes) (Assignment 2 on II-1 and II-2) -02/23
- II-2. Camera Calibration (slides) (lecture notes) –03/02 (in-person)
- II-3. Stereo Vision (slides) (Assignment 3 on II-3 and II-4), Project Ideas and Requirements – 03/09 (video with two parts with project ideas at the end of part 1; click >| beside the play button for part 2) (Project Idea Slide) (lecture notes ), 03/16 (video)
- II-4. Visual Motion – (slides) (lecture notes); Exam Quick Review (slides) – 03/23 (video), 03/30 (video with two parts) (Project Topic Proposal Due by March 28) – slides updated to here
Part III. Advanced Topics and Project Presentations
- III-1. Computer Vision for Social Good. (1) SAT-Hub: a Smart and Accessible Transportation Hub (SAT-Hub); (2) Human-Machine Perception and Assistive Technology – 04/06 (in-person)
- Mid-Term Exam in Class – 04/13 (Spring Recess: 04/15-04/22)
- III-2. People and Places: (1) ASSIST/iASSIST: Assistive Navigation Using a Smart Phone; (2) Storefront Accessibility Detection with Context Learning; (3) Posture Tracking and Localization for Social Distancing – 04/27 (with Exam Discussion)
- III-3. Vision and Visualization: (1) Crowd Analysis Using GANs and Regression; (2) Semantic Understanding with Self-Supervised Learning; (3) Large-scale 3D Visualization with Unity3D – 05/04
- All Student Project Presentations – 05/11; Project Reports by the end of 05/16 (Monday).
Textbook and References
- Computer Vision, In the form of Lecture Notes and Slides; will be provided by the instructor
- Vision and Brain – How We Perceive the World, By James V. Stone, The MIT Press. Paperback | $30.00 | ISBN: 9780262517737 | 264 pp. | 6 x 9 in | 25 color illus., 132 b&w illus.| September 2012 (For students with little experience in vision and neuroscience to know human vision, brain and computational neuroscience)
Online References and additional readings when necessary.
Grading and Prerequisites
The course will accommodate both PhD students in Computer Science and master level graduate students in Data Science and Cognitive Neuroscience at the CUNY Graduate Center. Students who take the course for credits will be required to finish 3 assignments (15% for each), one midterm exam (30%), and one programming project (25%, including submit a report (10%) and give a presentation (15%) to the class at the end of the semester). The topics of the projects will be given in the middle of the semester and will be related to the material presented in the lectures.
For the assignments and the projects, students may discuss ideas together. But since each student get credits for his or her submissions, all actual program code and written answers must be done individually by each student, and must not be shared. The midterm exam will be a close-book exam. You will need to clear state that you will neither give nor receive unauthorized assistance on this exam.
We fully support CUNY’s policy on Academic Honesty, which states, in part:
Academic dishonesty is unacceptable and will not be tolerated. Cheating, forgery, plagiarism, and collusion in dishonest acts undermine the CUNY’s educational mission and the students’ personal and intellectual growth. Students are expected to bear individual responsibility for their work, to learn the rules and definitions that underlie the practice of academic integrity, and to uphold its ideals. Ignorance of the rules is not an acceptable excuse for disobeying them. Any student who attempts to compromise or devalue the academic process will be sanctioned.
Academic sanctions in this class will range from an F on an assignment to an F in this course.
Students are required to have a good preparation in both mathematics (linear algebra/numerical analysis) and advanced programming.