Object based vehicle localization

Patent metadata

Field	Value
Publication number	US-12397799-B2
Application number	US-202217974649-A
Country	US
Kind code	B2
Filing date	Oct 27, 2022
Priority date	Oct 27, 2022
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

A method of self-localizing with respect to surrounding objects, comprising obtaining an approximated geolocation of the vehicle, retrieving mapping data comprising a geolocation of one or more stationary objects located in an area surrounding the approximated geolocation, receiving imagery data of a surrounding environment of the vehicle captured by a plurality of distinct imaging sensors deployed in the vehicle, applying one or more trained machine learning models to identify one or more of the stationary objects in the imagery data, computing a relative positioning of the vehicle with respect to one or more of the stationary objects based on an orientation of each of the plurality of imaging sensors with respect to the stationary object(s), computing an absolute positioning of the vehicle based on the relative positioning and the geolocation of the stationary object(s), and outputting the vehicle's absolute positioning.

First claim

What is claimed is:

1. A method of self-localizing with respect to surrounding objects, comprising:
capturing, by a plurality of distinct imaging sensors deployed in a vehicle, imagery data of a surrounding environment of said vehicle;
using at least one processor of a positioning system installed in said vehicle, said positioning system comprising said at least one processor, at least one storage device and a wireless communication interface, for:
obtaining from at least one navigation system deployed in said vehicle an approximated geolocation of the vehicle;
retrieving, from the at least one storage device and/or from at least one remote network resource accessed by said at least one processor through said wireless communication interface over a wireless communication network, mapping data comprising a geolocation of at least one stationary object located in an area surrounding the approximated geolocation;
receiving from said plurality of distinct imaging sensors said imagery data;
using at least one trained machine learning model to identify the at least one stationary object in the imagery data;
computing an orientation of each of the plurality of imaging sensors with respect to the at least one stationary object;
computing a relative positioning of the vehicle with respect to the at least one stationary object based on the computed orientation of said each of the plurality of imaging sensors with respect to the at least one stationary object;
computing a high-accuracy absolute positioning of the vehicle based on the relative positioning and the geolocation of at least one stationary object, wherein said absolute positioning is a geolocation of said vehicle including an elevation of the vehicle, with an accuracy higher than said approximated geolocation of said vehicle; and
outputting the computed high-accuracy vehicle's absolute positioning to at least one device deployed in said vehicle for navigation and/or controlling driving of said vehicle.

2. The method of claim 1 , wherein the orientation of each imaging sensor is expressed by a yaw, a pitch, and a roll of the respective imaging sensor with respect to the at least one stationary object.

3. The method of claim 1 , wherein the absolute positioning further comprises an orientation of the vehicle.

4. The method of claim 1 , wherein the at least one processor is further configured to compute the absolute positioning of the vehicle by:
computing a magnitude of at least one physical feature relating to the at least one stationary object based on analysis of the imagery data captured by at least one of the imaging sensors,
computing a relative positioning of the at least one imaging sensor with respect to the at least one stationary object based on the magnitude of the at least one physical feature, and
computing the absolute positioning of the vehicle based on the relative positioning of the at least one imaging sensor and the geolocation of the at least one stationary object.

5. The method of claim 1 , further comprising updating the absolute positioning of the vehicle based on the relative positioning of the vehicle with respect to at least one another stationary object identified in the imagery data captured by at least some of the plurality imaging sensors.

6. The method of claim 1 , wherein a positioning of each of the plurality of imaging sensors is calibrated with respect to the vehicle.

7. The method of claim 1 , wherein the surrounding environment comprises at least one member of a group consisting of: an outdoor environment, and an indoor environment.

8. The method of claim 1 , wherein the at least one stationary object is a member of a group consisting of: an infrastructure element, and a structure element.

9. The method of claim 1 , wherein the at least one machine learning model is trained to identify the at least one stationary object using a plurality of training samples associating between imagery data depicting the at least one stationary object and a label of the at least one stationary object.

10. The method of claim 1 , further comprising correlating the at least one stationary object identified in the imagery data captured by each of the plurality of imaging sensors based on a probability score computed by the at least one trained machine learning model for the identification of the respective at least one stationary object in the imagery data captured by each imaging sensors.

11. The method of claim 1 , further comprising updating the absolute positioning of the vehicle which dynamically moves based on new imagery data captured by at least one of the plurality of imaging sensors while and/or after the vehicle moves to a different location.

12. The method of claim 1 , wherein the approximated geolocation is derived from satellite navigation data captured by at least one satellite navigation sensor deployed in the vehicle.

13. The method of claim 1 , wherein the approximated geolocation is computed based on dead reckoning navigation data received from at least one dead reckoning navigation system of the vehicle.

14. The method of claim 1 , wherein the mapping data is locally stored in at least one non-transitory storage medium deployed in the vehicle.

15. The method of claim 1 , wherein the mapping data is received from at least one remote resource via at least one wireless communication channel established between the vehicle and the at least one remote resource.

16. A system for self-localizing with respect to surrounding objects, comprising:
a plurality of distinct imaging sensors deployed in a vehicle and configured to capture imagery data of a surrounding environment of said vehicle;
a positioning system installed in said vehicle, said positioning system comprising at least one processor, at least one storage device and a wireless communication interface;
the at least one processor configured to execute a code, the code comprising:
code instructions to obtain from at least one navigation system deployed in said vehicle an approximated geolocation of the vehicle;
code instructions to retrieve, from the at least one storage device and/or from at least one remote network resource accessed by said at least one processor through said wireless communication interface over a wireless communication network, mapping data comprising a geolocation of at least one stationary object located in an area surrounding the approximated geolocation;
code instructions to receive from said plurality of distinct imaging sensors said imagery data;
code instructions to use at least one trained machine learning model to identify the at least one stationary object in the imagery data;
code instructions to compute an orientation of each of the plurality of imaging sensors with respect to the at least one stationary object;
code instructions to compute a relative positioning of the vehicle with respect to the at least one stationary object based on the computed orientation of said each of the plurality of imaging sensors with respect to the at least one stationary object;
code instructions to compute a high-accuracy absolute positioning of the vehicle based on the relative positioning and the geolocation of at least one stationary object, wherein said absolute positioning is a geolocation of said vehicle including an elevation of the vehicle, with an accuracy higher than said approximated geolocation of said vehicle; and
code instructions to output the computed high-accuracy vehicle's absolute positioning to at least one device deployed in said vehicle for navigation and/or controlling driving of said vehicle.

Assignees

• Nec Corp America

Inventors

• Hoch Yaacov
• Bolgar Daniel
• Badikhi Yoav
• Lev Tsvi

Classifications

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• B60W2556/45

  External transmission of data to or from the vehicle · CPC title

• B60W2554/20

  Static objects · CPC title

• B60W2556/40

  High definition maps · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title