Method, apparatus, and system for generating feature correspondence from camera geometry

Patent metadata

Field	Value
Publication number	US-10949707-B2
Application number	US-201916287732-A
Country	US
Kind code	B2
Filing date	Feb 27, 2019
Priority date	Feb 27, 2019
Publication date	Mar 16, 2021
Grant date	Mar 16, 2021

Abstract

An approach is provided for determining a feature correspondence based on camera geometry. The approach, for example, involves determining a first labeled or detected pixel location in a first image and a second labeled or detected pixel location in a second image. The approach also involves computing a first ray from a first camera position of the first image through the first labeled or detected pixel location. The approach further involves computing a second ray from a second camera position of the second image through the second labeled or detected pixel location. The approach further involves computing a closeness value of the first ray and the second ray. The approach further involves providing an output indicating the feature correspondence between the first labeled or detected pixel location and the second labeled or detected pixel location based on determining that the closeness value is within a threshold value.

First claim

What is claimed is:

1. A computer-implemented method for determining a feature correspondence between a first image and a second image based on camera geometry comprising:
determining a first labeled or detected pixel location in the first image and a second labeled or detected pixel location in the second image;
computing a first ray from a first camera position of the first image through the first labeled or detected pixel location;
computing a second ray from a second camera position of the second image through the second labeled or detected pixel location;
computing a closeness value of the first ray and the second ray; and
providing an output indicating the feature correspondence between the first labeled or detected pixel location and the second labeled or detected pixel location based on determining that the closeness value is within a threshold value.

2. The method of claim 1 , wherein the closeness value is determined by computing a length of a line segment between the first ray and the second ray.

3. The method of claim 2 , wherein the line segment is orthogonal to both the first ray and the second ray.

4. The method of claim 1 , further comprising:
providing another output indicating that first labeled or detected pixel location, the second labeled or detected pixel location, or a combination thereof has no correspondence based on determining that the closeness value exceeds the threshold value.

5. The method of claim 1 , further comprising:
initiating an iterative computation of a subsequent closeness value between each subsequent ray pair computed between each subsequent first labeled or detected pixel location in the first image and each subsequent second labeled or detected pixel location in the second image.

6. The method of claim 5 , wherein the iterative computation is stopped based on determining that there is no more of said each subsequent first labeled or detected pixel location in the first image, there is no more of said each subsequent second labeled or detected pixel location in the second image, or a combination thereof.

7. The method of claim 1 , wherein the first labeled or detected pixel location is a first real-world location computed based on the first camera position, and wherein the second labeled or detected pixel location is a second real-world location computed based on the second camera position.

8. The method of claim 1 , wherein the output indicating the feature correspondence is used for at least one of:
a triangulation of a map feature for map making;
a real-time sensing of an environmental obstacle;
a determination of driver road behavior;
a determination of a safe or drivable map area;
an estimation of motion or depth; and
an image stitching.

9. An apparatus for determining a feature correspondence between a first image and a second image based on camera geometry comprising:
at least one processor; and
at least one memory including computer program code for one or more programs,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,
determine a first labeled or detected pixel location in the first image and a second labeled or detected pixel location in the second image;
compute a first ray from a first camera position of the first image through the first labeled or detected pixel location;
compute a second ray from a second camera position of the second image through the second labeled or detected pixel location;
compute a closeness value of the first ray and the second ray; and
provide an output indicating the feature correspondence between the first labeled or detected pixel location and the second labeled or detected pixel location based on determining that the closeness value is within a threshold value.

10. The apparatus of claim 9 , wherein the closeness value is determined by computing a length of a line segment between the first ray and the second ray.

11. The apparatus of claim 10 , wherein the line segment is orthogonal to both the first ray and the second ray.

12. The apparatus of claim 9 , wherein the apparatus is further caused to:
provide another output indicating that first labeled or detected pixel location, the second labeled or detected pixel location, or a combination thereof has no correspondence based on determining that the closeness value exceeds the threshold value.

13. The apparatus of claim 9 , wherein the apparatus is further caused to:
initiate an iterative computation of a subsequent closeness value between each subsequent ray pair compute between each subsequent first labeled or detected pixel location in the first image and each subsequent second labeled or detected pixel location in the second image.

14. The apparatus of claim 13 , wherein the iterative computation is stopped based on determining that there is no more of said each subsequent first labeled or detected pixel location in the first image, there is no more of said each subsequent second labeled or detected pixel location in the second image, or a combination thereof.

15. A non-transitory computer-readable storage medium for determining a feature correspondence between a first image and a second image based on camera geometry, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform:
determining a first labeled or detected pixel location in the first image and a second labeled or detected pixel location in the second image;
computing a first ray from a first camera position of the first image through the first labeled or detected pixel location;
computing a second ray from a second camera position of the second image through the second labeled or detected pixel location;
computing a closeness value of the first ray and the second ray; and
providing an output indicating the feature correspondence between the first labeled or detected pixel location and the second labeled or detected pixel location based on determining that the closeness value is within a threshold value.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the closeness value is determined by computing a length of a line segment between the first ray and the second ray.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the line segment is orthogonal to both the first ray and the second ray.

18. The non-transitory computer-readable storage medium of claim 15 , wherein the apparatus is caused to further perform:
providing another output indicating that first labeled or detected pixel location, the second labeled or detected pixel location, or a combination thereof has no correspondence based on determining that the closeness value exceeds the threshold value.

19. The non-transitory computer-readable storage medium of claim 15 , wherein the apparatus is caused to further perform:
initiating an iterative computation of a subsequent closeness value between each subsequent ray pair compute between each subsequent first labeled or detected pixel location in the first image and each subsequent second labeled or detected pixel location in the second image.

20. The non-transitory computer-readable storage medium of claim 19 , wherein the iterative computation is stopped based on determining that there is no more of said each subsequent first labeled or detected pixel location in the first image, there is no more of said each subsequent second labeled or

Assignees

• Here Global Bv

Inventors

• Mittal Anish
• Lawlor David
• Balakrishnan Krishna
• Chen Zhanwei

Classifications

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G01C21/3602 Primary

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G01C21/30

  Map- or contour-matching · CPC title

• G06T7/60

  Analysis of geometric attributes · CPC title