Projection matrix from Fundamental matrix
camera-calibration, computer-vision, matlab, opencv
Solution
A good reference book is "Multiple View Geometry in Computer Vision" from Hartley and Zisserman. First, your formula for P is wrong. If you want the formula with K inside, it is rather
P = K * [R | t]
or
P = [ [e']x * F | e']
but not a mix of both.
If you computed F from the 8 points algorithm, then you can recover only projective geometry up to a 3D homography (i.e. a 4x4 transformation).
To upgrade to euclidian space, there are 2 possibilities, both starting by computing the essential matrix.
First possibility is to compute the essential matrix from F: E = transpose(K2)*F*K1.
Second possibility, is to estimate directly the essential matrix for these 2 views:
- Normalize your 2D points by pre multiplying with inverse of K for each image ("normalized image coordinates")
- Apply the (same than for F) 8 points algorithm on these normalized points
- Enforce the fact that the essential matrix has its 2 singular values equal to 1 and last is 0, by SVD decomposition and forcing the diagonal values.
Once you have the essential matrix, we can compute the projection matrix in the form
P = K * [R | t]
R and t can be found thanks to the elements of the SVD of E (cf the previously mentioned book). However, you will have 4 possibilities. Only one of them projects points in front of both cameras, so you shall test a point (if you are sure of it) to remove the ambiguity among the 4. And in this case you will be able to place the camera and its orientation (with R and t of the projection) in your 3D scene.
Not so obvious, indeed...
Problem
I have obtained fundamental matrix between two cameras. I also, have their internal parameters in a `3 X 3` matrix which I had obtained earlier through chess board. Using the fundamental matrix, I have obtained `P1` and `P2` by `P1 = [I | 0]` and `P2 = [ [e']x * F | e']` These projection matrices are not really useful in getting the exact 3D location. Since, I have the internal parameters `K1` and `K2`, I changed `P1` and `P2` as `P1 = K1 * [I | 0]` and `P2 = K2 * [ [e']x * F | e']` - Is this the right way to get the real projection matrices which gives the actual relation between the 3D world and the image? - If not, please help me understand the right way and where I have gone wrong. - If this is the right approach, how do I verify these matrices?