Method and apparatus for localization using search space pruning

Patent metadata

Field	Value
Publication number	US-11790667-B2
Application number	US-202117144779-A
Country	US
Kind code	B2
Filing date	Jan 8, 2021
Priority date	Dec 20, 2018
Publication date	Oct 17, 2023
Grant date	Oct 17, 2023

Abstract

Methods described herein relate to reducing the computational intensity of vision-based localization. Methods may include: receiving sensor data from a vehicle traveling along a road; identifying one or more features of the environment from the sensor data; classifying the one or more identified features into one or more of a plurality of semantic classifications for the features; identifying map image data based on an identified location of the vehicle; identifying one or more features in the map image data; comparing one or more identified features of a first semantic classification with one or more features of the map image data of the first semantic classification; and registering a localized location of the vehicle within the environment based, at least in part, on the one or more identified features of the first semantic classification corresponding to the one or more features of the map image data of the first semantic classification.

First claim

That which is claimed:

1. An apparatus comprising at least one processor and at least one non-transitory memory including computer program code instructions, the computer program code instructions configured to, when executed, cause the apparatus to at least:
receive sensor data corresponding to an identified location in an environment;
identify one or more features of the environment from the sensor data;
classify the one or more features into one or more semantic classifications;
identify map data corresponding to the identified location, wherein the map data comprises a plurality of map features;
perform a comparison of the one or more features to the plurality of map features using a semantic category constraint; and
register a localized location within the environment based, at least in part, on the comparison.

2. The apparatus of claim 1 , wherein causing the apparatus to perform the comparison of the one or more features to the plurality of map features using the semantic category constraint comprises causing the apparatus to:
perform a comparison of at least one of the one or more features of the environment classified into a first semantic classification with at least one of the plurality of map features classified into the first semantic classification.

3. The apparatus of claim 2 , wherein causing the apparatus to perform the comparison of at least one of the one or more features of the environment classified into the first semantic classification with at least one of the plurality of map features classified into the first semantic classification comprises causing the apparatus to:
perform a comparison of the at least one of the one or more features of the environment classified into the first semantic classification with only the at least one of the plurality of map features classified into the first semantic classification.

4. The apparatus of claim 1 , wherein causing the apparatus to identify the map data corresponding to the identified location, comprises causing the apparatus to:
identify the map data corresponding to the identified location, where the map data comprises the plurality of map features classified into the one or more semantic classifications.

5. The apparatus of claim 1 , wherein causing the apparatus to register the localized location within the environment based, at least in part, on the comparison comprises causing the apparatus to:
register the localized location within the environment based, at least in part, on at least one of the one or more features of the environment classified into at least one of the one or more semantic classifications corresponding to at least one of the plurality of map features classified into the at least one of the one or more semantic classifications.

6. The apparatus of claim 1 , wherein the localized location is more accurate than the identified location.

7. The apparatus of claim 1 , wherein the semantic category constraint comprises constraining a comparison of the one or more features to map features having a same semantic classification.

8. The apparatus of claim 1 , wherein the semantic classifications include one or more of lane line type, lane line color, road sign type, or road sign color.

9. The apparatus of claim 1 , wherein causing the apparatus to register the localized location within the environment based, at least in part, on the comparison comprises causing the apparatus to:
identify map features of the plurality of map features corresponding to the one or more features based on the semantic category constraint; and
identify the localized location based on correspondences between the map features of the plurality of map features corresponding to the one or more features based on the semantic category constraint.

10. A computer program product having at least one non-transitory computer-readable storage medium with computer-executable program code instructions stored therein, the computer-executable program code instructions comprising program code instructions to:
receive sensor data corresponding to an identified location in an environment;
identify one or more features of the environment from the sensor data;
classify the one or more features into one or more semantic classifications;
identify map data corresponding to the identified location, wherein the map data comprises a plurality of map features;
perform a comparison of the one or more features to the plurality of map features using a semantic category constraint; and
register a localized location within the environment based, at least in part, on the comparison.

11. The computer program product of claim 10 , wherein the program code instructions to perform the comparison of the one or more features to the plurality of map features using the semantic category constraint comprise program code instructions to:
perform a comparison of at least one of the one or more features of the environment classified into a first semantic classification with at least one of the plurality of map features classified into the first semantic classification.

12. The computer program product of claim 11 , wherein the program code instructions to perform the comparison of at least one of the one or more features of the environment classified into the first semantic classification with at least one of the plurality of map features classified into the first semantic classification comprise program code instructions to:
perform a comparison of the at least one of the one or more features of the environment classified into the first semantic classification with only the at least one of the plurality of map features classified into the first semantic classification.

13. The computer program product of claim 10 , wherein the program code instructions to identify the map data corresponding to the identified location, comprise program code instructions to:
identify the map data corresponding to the identified location, where the map data comprises the plurality of map features classified into the one or more semantic classifications.

14. The computer program product of claim 10 , wherein the program code instructions to register the localized location within the environment based, at least in part, on the comparison comprise program code instructions to:
register the localized location within the environment based, at least in part, on at least one of the one or more features of the environment classified into at least one of the one or more semantic classifications corresponding to at least one of the plurality of map features classified into the at least one of the one or more semantic classifications.

15. The computer program product of claim 10 , wherein the localized location is more accurate than the identified location.

16. The computer program product of claim 10 , wherein the semantic category constraint comprises constraining a comparison of the one or more features to map features having a same semantic classification.

17. The computer program product of claim 10 , wherein the semantic classifications include one or more of lane line type, lane line color, road sign type, or road sign color.

18. The computer program product of claim 10 , wherein the program code instructions to register the localized location within the environment based, at least in part, on the comparison comprise program code instructions to:
identify map features of the plurality of map features corresponding to the one or more features based on the semantic category constraint; and
identify the localized location based on correspondenc

Assignees

• Here Global Bv

Inventors

• Viswanathan Anirudh

Classifications

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06V10/75

  Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries · CPC title

• G06V20/582

  of traffic signs · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title