Asynchronous observation matching for object localization in connected vehicles

Patent metadata

Field	Value
Publication number	US-11564116-B2
Application number	US-202016886560-A
Country	US
Kind code	B2
Filing date	May 28, 2020
Priority date	May 28, 2020
Publication date	Jan 24, 2023
Grant date	Jan 24, 2023

Abstract

The disclosure includes embodiments for asynchronous observation matching for object localization in connected vehicles. A method includes receiving a wireless message from the remote connected vehicle. The wireless message includes first observation data recorded by the remote connected vehicle and describing sensor measurements of the remote connected vehicle at a first time but not a second time. The method includes analyzing a set of vectors describing how a known object moves within a two-dimensional space to determine a two-dimensional flow of the known object in the two-dimensional space during a timespan. The method includes analyzing the two-dimensional flow of the object to infer a three-dimensional flow of the object in a three-dimensional space during the timespan. The method includes estimating second observation data recoded by the remote connected vehicle at the second time so that the impact of the latency is modified.

First claim

What is claimed is:

1. A method for modifying an impact of latency in wireless communications between an ego connected vehicle and a remote connected vehicle, the method executed by the ego connected vehicle and comprising:
receiving a wireless message from the remote connected vehicle, wherein the wireless message includes first observation data recorded by the remote connected vehicle and describing sensor measurements of the remote connected vehicle at a first time but not a second time;
analyzing a set of vectors describing how a known object moves within a two-dimensional space to determine a two-dimensional flow of the known object in the two-dimensional space during a timespan from the first time to the second time, wherein the set of vectors is determined based on the observation data of the remote connected vehicle from the first time and sensor measurements of the ego connected vehicle at the first time and the second time;
analyzing the two-dimensional flow of the object to infer a three-dimensional flow of the object in a three-dimensional space during the timespan; and
estimating, based on the three-dimensional flow of the object in the three-dimensional space, second observation data recoded by the remote connected vehicle at the second time, wherein the estimate occurs prior to receipt of the second observation data from the remote connected vehicle by the ego connected vehicle due to latency so that the impact of the latency is modified.

2. The method of claim 1 , wherein the set of vectors are represented by a set of hypergraphs.

3. The method of claim 1 , wherein the three-dimensional space is real-world.

4. The method of claim 3 , wherein the real-world includes the ego connected vehicle, the remote connected vehicle, and the object.

5. The method of claim 1 , wherein the wireless message is a cellular-vehicle-to-everything message.

6. The method of claim 1 , further comprising determining a location of the object in the three-dimensional space based on the second observation data which is estimated and the sensor measurements of the ego vehicle at the second time.

7. The method of claim 6 , wherein the location is accurate to within plus or minus three meters relative to an actual location of the object in the three-dimensional space.

8. The method of claim 1 , wherein the ego vehicle is a hub vehicle of a vehicular micro cloud and the method improves the impact of latency among the wireless communications among members of the vehicular micro cloud.

9. The method of claim 8 , wherein the hub vehicle controls membership in the vehicular micro cloud and can instruct the members when to join and when to leave the vehicular micro cloud.

10. A computer program product included in an onboard vehicle computer of an ego connected vehicle for modifying an impact of latency in wireless communications between the ego connected vehicle and a remote connected vehicle, the computer program product comprising computer code that is operable, when executed by the onboard computer, to cause the onboard computer to execute steps including:
receiving a wireless message from the remote connected vehicle, wherein the wireless message includes first observation data recorded by the remote connected vehicle and describing sensor measurements of the remote connected vehicle at a first time but not a second time;
analyzing a set of vectors describing how a known object moves within a two-dimensional space to determine a two-dimensional flow of the known object in the two-dimensional space during a timespan from the first time to the second time, wherein the set of vectors is determined based on the observation data of the remote connected vehicle from the first time and sensor measurements of the ego connected vehicle at the first time and the second time;
analyzing the two-dimensional flow of the object to infer a three-dimensional flow of the object in a three-dimensional space during the timespan; and
estimating, based on the three-dimensional flow of the object in the three-dimensional space, second observation data recoded by the remote connected vehicle at the second time, wherein the estimate occurs prior to receipt of the second observation data from the remote connected vehicle by the ego connected vehicle due to latency so that the impact of the latency is modified.

11. The computer program product of claim 10 , wherein the set of vectors are represented by a set of hypergraphs.

12. The computer program product of claim 10 , wherein the three-dimensional space is real-world.

13. The computer program product of claim 12 , wherein the real-world includes the ego connected vehicle, the remote connected vehicle, and the object.

14. The computer program product of claim 10 , wherein the wireless message is a cellular-vehicle-to-everything message.

15. A system included in an ego connected vehicle for modifying an impact of latency in wireless communications between the ego connected vehicle and a remote connected vehicle, the system comprising:
a processor;
a communication unit communicatively coupled to the processor; and
a non-transitory memory communicatively coupled to the processor and the communication unit, wherein the non-transitory memory stores executable code that is operable, when executed by the processor, to cause the processor to:
receive a wireless message from the remote connected vehicle, wherein the wireless message includes first observation data recorded by the remote connected vehicle and describing sensor measurements of the remote connected vehicle at a first time but not a second time;
analyze a set of vectors describing how a known object moves within a two-dimensional space to determine a two-dimensional flow of the known object in the two-dimensional space during a timespan from the first time to the second time, wherein the set of vectors is determined based on the observation data of the remote connected vehicle from the first time and sensor measurements of the ego connected vehicle at the first time and the second time;
analyze the two-dimensional flow of the object to generate an inference of a three-dimensional flow of the object in a three-dimensional space during the timespan; and
estimate, based on the three-dimensional flow of the object in the three-dimensional space, second observation data recoded by the remote connected vehicle at the second time, wherein the estimate occurs prior to receipt of the second observation data from the remote connected vehicle by the ego connected vehicle due to latency so that the impact of the latency is modified.

16. The system of claim 15 , wherein the wireless message is a cellular-vehicle-to-everything message.

17. The system of claim 15 , further comprising determining a location of the object in the three-dimensional space based on the second observation data which is estimated and the sensor measurements of the ego vehicle at the second time.

18. The system of claim 17 , wherein the location is accurate to within plus or minus three meters relative to an actual location of the object in the three-dimensional space.

19. The system of claim 15 , wherein the ego vehicle is a hub vehicle of a vehicular micro cloud.

20. The system of claim 15 , wherein the executable code is determined based at least in part on a digital twin simulation of the timespan that provides digital data for analyzing the two-dimensional flow of the object to infer the three-dimensional flow of the object in the three-dimensional space during the timespan, wherein

Assignees

• Toyota Eng & Mfg North America

Inventors

• Guo Rui
• Lu Hongsheng
• Gao Peng

Classifications

• H04W4/029

  Location-based management or tracking services · CPC title

• H04W4/40

  for vehicles, e.g. vehicle-to-pedestrians [V2P] · CPC title

• H04W24/08 Primary

  Testing, {supervising or monitoring} using real traffic · CPC title

• H04W4/46 Primary

  for vehicle-to-vehicle communication [V2V] · CPC title

• H04W4/44

  for communication between vehicles and infrastructures, e.g. vehicle-to-cloud [V2C] or vehicle-to-home [V2H] · CPC title