Light-based object localization

Patent metadata

Field	Value
Publication number	US-12347129-B2
Application number	US-202217697968-A
Country	US
Kind code	B2
Filing date	Mar 18, 2022
Priority date	Mar 18, 2022
Publication date	Jul 1, 2025
Grant date	Jul 1, 2025

Abstract

Provided are methods for light-based object localization, which can include comparing unexpected light sources to expected light sources for determination of an agent, such as a partially and/or fully occluded agent. Some methods described also include generating a trajectory for an autonomous vehicle based on the comparison. Systems and computer program products are also provided.

First claim

What is claimed is:

1. A method comprising:
obtaining, using at least one processor, sensor data associated with an environment in which an autonomous vehicle is operating;
identifying, using the at least one processor, at least one light source in the environment based on the sensor data;
determining, using the at least one processor, scene data based on the sensor data associated with the environment, wherein the scene data includes one or more predetermined light sources in the environment;
determining, using the at least one processor, whether the at least one light source corresponds to at least one of the one or more predetermined light sources associated with the scene data;
in response to determining that the at least one light source does not correspond to at least one of the one or more predetermined light sources, identifying, using the at least one processor, the at least one light source as at least one unexpected light source in the environment;
identifying, using the at least one processor, an agent in the environment based on the at least one unexpected light source;
generating, using the at least one processor, a trajectory for the autonomous vehicle based on the identified agent; and
controlling operation of the autonomous vehicle based on the generated trajectory.

2. The method of claim 1 , wherein identifying the agent in the environment comprises determining, using the at least one processor, based on the sensor data and the at least one unexpected light source, an agent location in the environment.

3. The method of claim 1 , wherein identifying the agent comprises:
determining, using the at least one processor, based on the at least one unexpected light source, a candidate location associated with a candidate agent;
generating, using the at least one processor, based on the scene data and the at least one unexpected light source, a light propagation result for the candidate agent at the candidate location; and
identifying, using the at least one processor, based on the light propagation result, the agent.

4. The method of claim 3 , wherein generating the light propagation result for the candidate agent at the candidate location is based on a type of agent.

5. The method of claim 4 , wherein the type of agent includes one or more of: a vehicle, a car, a motorcycle, a pedestrian, and a bicycle.

6. The method of claim 1 , wherein identifying the agent comprises:
determining, using the at least one processor, based on the at least one unexpected light source, a predictive candidate location associated with a candidate agent;
generating, using the at least one processor, based on the scene data and the at least one unexpected light source, a reverse light tracing result at the predictive candidate location; and
identifying, using the at least one processor, based on the reverse light tracing result, the agent.

7. The method of claim 1 , wherein identifying the agent comprises determining, using the at least one processor, based on the sensor data and the at least one unexpected light source, an agent trajectory parameter indicative of a trajectory of the agent.

8. The method of claim 1 , wherein identifying the at least one light source comprises:
determining, using the at least one processor, based on the sensor data and the scene data, a differential scene indicative of differences in light intensity between the scene data and the sensor data; and
identifying the at least one light source is based on the differential scene.

9. The method of claim 1 , wherein the sensor data is obtained from one or more of: a camera, a light-intensity sensor, and a LIDAR sensor.

10. The method of claim 1 , wherein the scene data includes one or more of: a location parameter indicative of a location of the autonomous vehicle, a time parameter indicative of a time of day, and a weather parameter indicative of a weather condition of the environment.

11. The method of claim 1 , wherein the scene data comprises a three-dimensional scene data.

12. The method of claim 1 , wherein determining the scene data comprises determining, based on one or more of a stereoscopic scene builder, a LIDAR scene builder, and a sensor fusion scene builder, the scene data.

13. The method of claim 1 , further comprising:
generating, using the at least one processor, based on the agent, an advance warning indication.

14. A non-transitory computer readable medium comprising instructions stored thereon that, when executed by at least one processor, cause the at least one processor to carry out operations comprising:
obtaining, using at least one processor, sensor data associated with an environment in which an autonomous vehicle is operating;
identifying, using the at least one processor, at least one light source in the environment based on the sensor data;
determining, using the at least one processor, scene data based on the sensor data associated with the environment, wherein the scene data includes one or more predetermined light sources in the environment;
determining, using the at least one processor, whether the at least one light source corresponds to at least one of the one or more predetermined light sources associated with the scene data; in response to determining that the at least one light source does not correspond to at least one of the one or more predetermined light sources, identifying, using the at least one processor, the at least one light source as an at least one unexpected light source in the environment;
identifying, using the at least one processor, an agent in the environment based on the at least one unexpected light source;
generating, using the at least one processor, a trajectory for the autonomous vehicle based on the identified agent; and
controlling, using the at least one processor, operation of the autonomous vehicle based on the generated trajectory.

15. A system, comprising at least one processor; and at least one memory storing instructions thereon that, when executed by the at least one processor, cause the at least one processor to:
obtain sensor data associated with an environment in which an autonomous vehicle is operating;
identify at least one light source in the environment based on the sensor data;
determine scene data based on the sensor data associated with the environment, wherein the scene data includes one or more predetermined light sources in the environment;
determine whether the at least one light source corresponds to at least one of the one or more predetermined light sources associated with the scene data;
in response to determining that the at least one light source does not correspond to at least one of the one or more predetermined light sources, identify the at least one light source as at least one unexpected light source in the environment;
identify an agent in the environment based on the at least one unexpected light source;
generate a trajectory for the autonomous vehicle based on the identified agent; and
control operation of the autonomous vehicle based on the generated trajectory.

Assignees

• Motional Ad Llc

Inventors

• Cserna Bence
• Singhal Puneet

Classifications

• B60W2420/408

  Radar; Laser, e.g. lidar · CPC title

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2200/04

  involving 3D image data · CPC title