Computer Vision Spring 2022 – Assignment 3 – CUNY Computational Vision and Convergence Laboratory

Computer Science – The CUNY Graduate Center
Computer Vision and Image Processing
Assignment 3 ( Deadline: 4/12 Tuesday before midnight– extended to 04/19/2022 Tuesday midnight)
===============================================================
Note: All the writings of your assignment must be in “soft” copies (in a single PDF file) by sending to Prof. Zhu <cv.zhu.ccny@gmail.com> via an email attachment. You are responsible for the lose of your submissions if you don’t include “GC 74030-1″ (exactly) in the subject of your email. For your programming part, in addition to the writing report, please also send your source code – in their original formats; please don’t format them into PDF or Word formats. Please don’t send in your images and executable. Please do not zip your files into one zipped file! You may want to include images, tables, etc in your report as they show results of your work (Those marked with * are optional for extra credits)

Please don’t forget to write your name and ID (last four digits) in both your report and the code, right after the title (if any) of your report. Then under your name, please write this statement:
“The work in this assignment is my own. Any outside sources have been properly cited.”
Without writing this statement, you will not be able to get any score.

1. (Stereo- 20 points ) Estimate the accuracy of the simple stereo system (Figure 3 in the lecture notes of stereo vision) assuming that the only source of noise is the localization of corresponding points in the two images. Please derive (12 points) and discuss (8 points) the dependence of the error in depth estimation of a 3D point as a function of (1) the baseline width, (2) the focal length, (3) stereo matching error, and (4) the depth of the 3D point.

Hint: D = f B/d; Take the partial derivatives of D with respect to the disparity d.

2. (Motion- 20 points) Could you obtain 3D information of a scene by viewing the scene by using multiple frames of images taken by a camera rotating around its optical center (5 points)? Discuss why or why not(5 points). What about translating (moving, not zooming!) the camera along the direction of its optical axis (5 points)? Explain. (5 points)

3. (Motion- 10 points) (1) Explain what is the aperture problem, and how it can be solved if a corner is visible through the aperture (10 pts).

4. (Stereo and Motion – 10 points): (1) Give 5 examples when humans using stereo or motion in daily life or work (5 points) (2) Give another 5 examples that use computer vision techniques with stereo or motion in real applications.

5. (Stereo Programming – 40 points + 5 bonus points ) Use the image pair ( Image 1, Image 2) for the following exercises.

(1). Fundamental Matrix. – Design and implement a program that, given a stereo pair, determines at least eight point matches, then recovers the fundamental matrix (5 points ) and the location of the epipoles (5 points). Check the accuracy of the result by measuring the distance between the estimated epipolar lines and image points not used by the matrix estimation (5 points). Also, overlay the epipolar lines of control points and test points on one of the images (say Image 1- I already did this in the starting code below). Control points are the correspondences (matches) used in computing the fundamental matrix, and test points are those used to check the accuracy of the computation.

Hint: As a first step, you can pick up the matches of both the control points and the test points manually. You may use my matlab code (FmatGUI.m) as a starting point – where I provided an interface to pick up point matches by mouse clicks. The epipolar lines should be (almost) parallel in this stereo pair. If not, something is wrong either with your code or the point matches. Make sure this is achieved before you move to the second step* – that is to try to search for point matches automatically by your program. However the second step is optional (for extra 5 points)

(2). Feature-based matching. – Design a stereo vision system to do “feature-based matching” and explain your algorithm in writing – what the feature is, how effect it is, and what are the problems (5 points). The system should have a user interface that allows a user to select a point on the first image, say by a mouse click (5 points). The system should then find and highlight the corresponding point on the second image, say using a cross hair points). Try to use the epipolar geometry derived from (1) in searching correspondences along epipolar lines (5 points).

Hint : You may use a similar interface as I did for question (1). You may use the point match searching algorithm in (1) (if you have done so), but this time you need to constrain your search windows along the epipolar lines.

(3) Discussions. Show your results on points with different properties like those in corners, edges, smooth regions, textured regions, and occluded regions that are visible only in one of the images. Discuss for each case, why your vision system succeeds or fails in finding the correct matches (5 points). Compare the performance of your system against a human user (e.g. yourself) who marks the corresponding matches on the second image by a mouse click (5 points).