Crowdsourcing a sparse map for autonomous vehicle navigation

Patent metadata

Field	Value
Publication number	US-11086334-B2
Application number	US-201715656223-A
Country	US
Kind code	B2
Filing date	Jul 21, 2017
Priority date	Jul 21, 2016
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

Systems and methods are provided for crowdsourcing a sparse map for autonomous vehicle navigation. In one implementation, a non-transitory computer-readable medium may include a sparse map for autonomous vehicle navigation along a road segment. The sparse map may include at least one line representation of a road surface feature extending along the road segment, each line representation representing a path along the road segment substantially corresponding with the road surface feature, and wherein the road surface feature is identified through image analysis of a plurality of images acquired as one or more vehicles traverse the road segment and a plurality of landmarks associated with the road segment.

First claim

What is claimed is:

1. A non-transitory computer-readable medium including a sparse map for autonomous vehicle navigation along a road segment, the sparse map comprising:
at least one line representation of a road surface feature extending along the road segment, each line representation comprising at least one polynomial representation of a path along the road segment substantially corresponding with the road surface feature, wherein:
the road surface feature is identified through image analysis of a plurality of images acquired as a plurality of vehicles traverse the road segment,
the at least one line representation is determined by aligning first data from the image analysis of the plurality of images captured by the plurality of vehicles, and
the first data comprises position information associated with the road surface feature;

a target trajectory generated based on the aligning of the first data; and
a plurality of landmarks associated with the road segment, wherein the landmarks are determined by aligning second data, the second data being obtained from further image analysis of the plurality of images captured by the plurality of vehicles.

2. The non-transitory computer-readable medium of claim 1 , wherein the road surface feature includes at least one of a road edge or a lane marking.

3. The non-transitory computer-readable medium of claim 1 , wherein at least one of the plurality of landmarks includes a road sign.

4. The non-transitory computer-readable medium of claim 1 , wherein the plurality of landmarks are spaced apart by an average distance in the map of at least 50 m.

5. The non-transitory computer-readable medium of claim 1 , wherein the sparse map has a data density of no more than 1 megabyte per kilometer.

6. The non-transitory computer-readable medium of claim 1 , wherein the at least one line representation of the road surface feature further includes a spline, a polynomial representation, or a curve.

7. The non-transitory computer-readable medium of claim 1 , wherein the plurality of landmarks are identified through the further image analysis of the plurality of images acquired as the plurality of vehicles traverse the road segment.

8. The non-transitory computer-readable medium of claim 7 , wherein the image analysis to identify the plurality of landmarks includes accepting potential landmarks when a ratio of images in which the landmark does appear to images in which the landmark does not appear exceeds a threshold.

9. The non-transitory computer-readable medium of claim 7 , wherein the image analysis to identify the plurality of landmarks includes rejecting potential landmarks when a ratio of images in which the landmark does not appear to images in which the landmark does appear exceeds a threshold.

10. The non-transitory computer-readable medium of claim 1 , wherein the at least one polynomial representation comprises a plurality of polynomial representations connected to form a spline.

11. The non-transitory computer-readable medium of claim 1 , wherein the at least one polynomial representation is represented by a plurality of parameters comprising the coefficients of a corresponding polynomial function.

12. The non-transitory computer-readable medium of claim 1 , wherein the at least one line representation has a data density of no more than 192 bytes per 100 meters.

13. The non-transitory computer-readable medium of claim 1 , wherein the sparse map further comprises one or more portions of metadata associated with the at least one line representation.

14. The non-transitory computer-readable medium of claim 13 , wherein the one or more portions describe a number of lanes associated with the at least one line representation.

15. The non-transitory computer-readable medium of claim 1 , wherein the plurality of images are each segmented into a plurality of patches such that the data alignment of the first and second data is performed within corresponding patches of each plurality of patches.

16. The non-transitory computer-readable medium of claim 1 , wherein the line representation is determined using a smooth line model on the first aligned data.

17. The non-transitory computer-readable medium of claim 1 , wherein the line representation is stored using a first set of coordinate axes and at least one some of the data from the plurality of images uses a second set of coordinate axes.

18. The non-transitory computer-readable medium of claim 1 , wherein aligning the first data comprises longitudinally or laterally shifting or stretching the at least one line representation.

19. The non-transitory computer-readable medium of claim 1 , wherein target trajectory is generated using data contained in the sparse map.

20. The non-transitory computer-readable medium of claim 1 , wherein the target trajectory is associated with a lane of the road segment.

21. A system for generating a sparse map for autonomous vehicle navigation along a road segment, comprising:
at least one processing device configured to:
receive a plurality of images acquired as a plurality of vehicles traverse the road segment;
identify, based on image analysis of the plurality of images, at least one line representation of a road surface feature extending along the road segment, each line representation comprising at least one polynomial representation of a path along the road segment substantially corresponding with the road surface feature, wherein:
the at least one line representation is identified by aligning first data from the image analysis of the plurality of images captured by the plurality of vehicles, and
the first data comprises position information associated with the road surface feature;

generate a target trajectory based on the aligning of the first data; and
identify, based on the plurality of images, a plurality of landmarks associated with the road segment, wherein the landmarks are identified by aligning second data, the second data being obtained from further image analysis of the plurality of images captured by the plurality of vehicles.

22. The system of claim 21 , wherein the road surface feature includes at least one of a road edge or a lane marking.

23. The system of claim 21 , wherein at least one of the plurality of landmarks includes a road sign.

24. The system of claim 21 , wherein the at least one line representation of the road surface feature further includes a spline, a polynomial representation, or a curve.

25. The system of claim 21 , wherein identifying the plurality of landmarks includes the further analysis of the plurality of images acquired as the plurality of vehicles traverse the road segment.

26. The system of claim 25 , wherein analyzing the plurality of images to identify the plurality of landmarks includes accepting potential landmarks when a ratio of images in which the landmark does appear to images in which the landmark does not appear exceeds a threshold.

27. The system of claim 25 , wherein analyzing the plurality of images to identify the plurality of landmarks includes rejecting potential landmarks when a ratio of images in which the landmark does not appear to images in which the landmark does appear exceeds a threshold.

28. A method for generating a sparse map for autonomous vehicle navigation along a road segment, comprising:
receiving a plurality of images acquired as a plurality of vehicles traverse

Assignees

• Mobileye Vision Technologies Ltd

Inventors

• Fridman Ofer

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G01C21/3852

  Data derived from aerial or satellite images · CPC title

• G01C21/3822

  Road feature data, e.g. slope data · CPC title

• G05D1/246

  using environment maps, e.g. simultaneous localisation and mapping [SLAM] · CPC title

• G05D1/2435

  Extracting 3D information · CPC title