Systems and methods for intelligently calibrating infrastructure devices using onboard sensors of an autonomous agent

Patent metadata

Field	Value
Publication number	US-11513189-B2
Application number	US-202117193621-A
Country	US
Kind code	B2
Filing date	Mar 5, 2021
Priority date	Feb 15, 2019
Publication date	Nov 29, 2022
Grant date	Nov 29, 2022

Abstract

A system for intelligently implementing an autonomous agent that includes an autonomous agent, a plurality of infrastructure devices, and a communication interface. A method for intelligently calibrating infrastructure (sensing) devices using onboard sensors of an autonomous agent includes identifying a state of calibration of an infrastructure device, collecting observation data from one or more data sources, identifying or selecting mutually optimal observation data, specifically localizing a subject autonomous agent based on granular mutually optimal observation data, identifying dissonance in observation data from a perspective of a subject infrastructure device, and recalibrating a subject infrastructure device.

First claim

What is claimed is:

1. A method for determining a calibration error associated with at least one of an infrastructure device and an autonomous agent, the method comprising:
receiving, at a computing system:
a first stream of observation data from a first set of sensors onboard the infrastructure device, wherein the infrastructure device is mounted to a stationary infrastructure component arranged in an operating environment of the autonomous agent, wherein the first set of sensors comprises a camera having a focal length, wherein the focal length is determined at least in part based on a distance between the infrastructure device comprising the camera and a nearest infrastructure device of the set of infrastructure devices;
a second stream of observation data from a second set of sensors onboard the autonomous agent, the autonomous agent remote from the infrastructure device; and

determining an association between the first stream of observation data and the second stream of observation data to determine a set of mutually optimal observation data;
localizing the autonomous agent based on the mutually optimal observation data;
determining a dissonance between the first and second streams of observation data based on the association;
determining a calibration error based on the dissonance; and
updating a state of calibration of at least one of the first and second sets of sensors based on the calibration error.

2. The method of claim 1 , further comprising removing at least one of noise and extraneous data from the set of mutually optimal observation data to determine a set of granular mutually optimal observation data.

3. The method of claim 2 , wherein localizing the autonomous agent is based on the granular mutually optimal observation data.

4. The method of claim 1 , wherein the infrastructure device is of a set of multiple infrastructure devices arranged in the operating environment of the autonomous agent.

5. The method of claim 1 , wherein at least a portion of the computing system is arranged remote from the autonomous agent and the infrastructure device.

6. The method of claim 1 , further comprising determining a source of the calibration error.

7. The method of claim 6 , wherein determining the source of the calibration error comprises receiving a third stream of observation data from a second infrastructure device of the set of infrastructure devices, wherein the source of the calibration error is determined at least in part based on the third stream of observation data.

8. A system for updating a state of calibration, the system comprising:
a set of infrastructure devices comprising a set of onboard sensor systems, wherein each of the set of infrastructure devices is mounted to a stationary infrastructure component arranged in an operating environment of an autonomous agent; and
a computing system in communication with the set of infrastructure devices and the autonomous agent, wherein the computing system:
receives a first stream of observation data from a first onboard sensor system of the set of onboard sensor systems, wherein the first onboard sensor system is onboard a first infrastructure device of the set of infrastructure devices;
receives a second stream of observation data from the autonomous agent;
determines an association between the first stream of observation data and the second stream of observation data to determine a set of mutually optimal observation data;
localizes the autonomous agent based on the mutually optimal observation data;
determines a dissonance between the first and second streams of observation data based on the association;
determines a calibration error based on the dissonance;
receives a third stream of observation data from a second infrastructure device of the set of infrastructure devices;
determines a source of the calibration error based on the third stream of observation data; and
updates a state of calibration of at least one of the first infrastructure device and the autonomous agent based on the calibration error and the source of the calibration error.

9. The system of claim 8 , wherein the computing system further removes at least one of noise and extraneous data from the set of mutually optimal observation data to determine a set of granular mutually optimal observation data.

10. The system of claim 9 , wherein the computing system localizes the autonomous agent based on the granular mutually optimal observation data.

11. The system of claim 8 , wherein at least a portion of the computing system is arranged remote from the autonomous agent and the infrastructure device, and wherein the set of infrastructure devices defines a field of sensing, wherein the autonomous agent is arranged within the field of sensing.

12. A system for updating a state of calibration, the system comprising:
a set of infrastructure devices comprising a set of onboard sensor systems, wherein each of the set of onboard sensor systems comprises a camera, wherein a focal length of the camera is determined at least in part based on a distance between the infrastructure device comprising the camera and a nearest infrastructure device of the set of infrastructure devices; and
a computing system in communication with the set of infrastructure devices and an autonomous agent, wherein the computing system:
receives a first stream of observation data from a first onboard sensor system of the set of onboard sensor systems, wherein the first onboard sensor system is onboard a first infrastructure device of the set of infrastructure devices;
receives a second stream of observation data from the autonomous agent;
determines an association between the first stream of observation data and the second stream of observation data;
determines a dissonance between the first and second streams of observation data based on the association;
determines a calibration error based on the dissonance; and
updates a state of calibration of at least one of the first infrastructure device and the autonomous agent based on the calibration error.

13. The system of claim 12 , wherein at least a portion of the computing system is arranged remote from the autonomous agent and the infrastructure device.

14. The system of claim 12 , wherein the set of infrastructure devices defines a field of sensing, wherein the autonomous agent is arranged within the field of sensing.

15. The system of claim 12 , wherein the computing system further determines a source of the calibration error.

16. The system of claim 15 , wherein the computing system further receives a third stream of observation data from a second infrastructure device of the set of infrastructure devices, wherein the source of the calibration error is determined at least in part based on the third stream of observation data.

Assignees

• May Mobility Inc

Inventors

• Voorheis Tom
• Goeddel Rob
• Vozar Steve
• Olson Edwin

Classifications

• G01S17/931

  of land vehicles · CPC title

• G01S13/867

  Combination of radar systems with cameras · CPC title

• G01S7/497

  Means for monitoring or calibrating · CPC title

• G02B7/40

  using time delay of the reflected waves, e.g. of ultrasonic waves · CPC title

• B60R11/04

  Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle · CPC title