Method and apparatus for providing real time feature triangulation

Patent metadata

Field	Value
Publication number	US-11188765-B2
Application number	US-201816209558-A
Country	US
Kind code	B2
Filing date	Dec 4, 2018
Priority date	Dec 4, 2018
Publication date	Nov 30, 2021
Grant date	Nov 30, 2021

Abstract

An approach is provided for generating a reverse sequence or real-time streamed images for triangulation. The approach includes receiving a real-time stream of images captured by a sensor of a vehicle during a drive; extracting a sequence of two or more images from the real-time stream; reversing the sequence of the two or more images; and providing the reversed sequence of the two or more images for feature triangulation.

First claim

What is claimed is:

1. A method for generating a re-arranged sequence of real-time streamed images for triangulation of a feature, comprising:
receiving a real-time stream of images captured by a sensor of a vehicle during a drive;
extracting a sequence of images from the real-time stream, wherein the images comprise at least a first image being captured later in time than a second image;
re-arranging the first image and second image in reverse time order based on respective image capture times; and
providing the re-arranged sequence of the images for the triangulation of the feature,
wherein a vehicle trajectory indicates that the vehicle is traveling towards the feature during the drive when the images are captured.

2. The method of claim 1 , further comprising:
caching a most recent designated number of images of the real-time stream in a cache memory,
wherein the re-arranged sequence of the images is extracted from the cache memory.

3. The method of claim 2 , wherein the caching further comprises updating the triangulation of the feature based on one or more additional images real-time streamed from the sensor.

4. The method of claim 1 , wherein the extracting of the sequence of the images is based on movement of the feature relative to a center of each of the images.

5. The method of claim 1 , wherein the re-arranged sequence of the images includes the first image including a first image size of the feature being larger than a second image size of the feature in the second image.

6. The method of claim 1 , wherein the feature triangulation is performed by in-vehicle hardware.

7. The method of claim 1 , further comprising:
receiving real-time stream of vehicle trajectory; and
performing the feature triangulation based on determining that the vehicle trajectory indicates that the vehicle has passed the feature or approached the feature to within a threshold distance.

8. The method of claim 7 , wherein the threshold distance is based on obtaining a target image size of the feature in one of the images.

9. The method of claim 8 , wherein the vehicle trajectory indicates that the vehicle is traveling towards the feature during the drive, and
the vehicle trajectory is a time-ordered sequence of location probe points determined by a location sensor of the vehicle.

10. An apparatus for generating a re-arranged sequence of real-time streamed images for triangulation of a feature comprising:
at least one processor; and
at least one memory including computer program code for one or more programs,
the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following,
receive a real-time stream of images captured by a sensor of a vehicle during a drive;
extract a sequence of images from the real-time stream, wherein the images comprise at least a first image being captured later in time than a second image;
re-arrange the first image and second image in reverse time order based on respective image capture times; and

provide the re-arranged sequence of the images for the triangulation of the feature,
wherein a vehicle trajectory indicates that the vehicle is traveling towards the feature during the drive when the images are captured.

11. The apparatus of claim 10 , wherein the apparatus is further caused to:
cache a most recent designated number of images of the real-time stream in a cache memory,
wherein the re-arranged sequence of the images is extracted from the cache memory.

12. The apparatus of claim 11 , wherein the apparatus is further caused to:
update the triangulation of the feature based on additional images real-time streamed from the sensor.

13. The apparatus of claim 10 , wherein the sequence of extracted images is based on movement of the feature relative to a center of each of the images.

14. The apparatus of claim 10 , wherein the re-arranged sequence of the images includes the first image including a first image size of the feature being larger than a second image size of the feature in the second image.

15. The apparatus of claim 10 , wherein the apparatus is further caused to:
receive real-time stream of vehicle trajectory; and
perform the feature triangulation based on determining that the vehicle trajectory indicates that the vehicle has passed the feature or approached the feature to within a threshold distance.

16. A non-transitory computer-readable storage medium for generating a re-arranged sequence of real-time streamed images for triangulation of a feature, carrying one or more sequences of one or more instructions which, when executed by one or more processors, cause an apparatus to perform:
receiving a real-time stream of images captured by a sensor of a vehicle during a drive;
extracting a sequence of images from the real-time stream, wherein the images comprise at least a first image being captured later in time than a second image;
re-arranging the first image and second image in reverse time order based on respective image capture times; and
providing the re-arranged sequence of the images for triangulation of the feature,
wherein a vehicle trajectory indicates that the vehicle is traveling towards the feature during the drive when the images are captured.

17. The non-transitory computer-readable storage medium of claim 16 , wherein the apparatus is further caused to perform:
caching a most recent designated number of images of the real-time stream in a cache memory,
wherein the re-arranged sequence of the images is extracted from the cache memory.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the apparatus is further caused to perform:
updating the triangulation of the feature based on additional images real-time streamed from the sensor.

19. The non-transitory computer-readable storage medium of claim 17 , wherein the sequence of extracted images is based on movement of the feature relative to a center of each of the images.

Assignees

• Here Global Bv

Inventors

• Viswanathan Anirudh

Classifications

• G06V10/757

  Matching configurations of points or features · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T7/97

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• G06T17/05 Primary

  Geographic models · CPC title

• G06T7/73

  using feature-based methods · CPC title