Systems and methods for multi-modal transfer capabilities for smart infrastructure

Patent metadata

Field	Value
Publication number	US-11709061-B2
Application number	US-202017073119-A
Country	US
Kind code	B2
Filing date	Oct 16, 2020
Priority date	Oct 16, 2020
Publication date	Jul 25, 2023
Grant date	Jul 25, 2023

Abstract

Systems, methods, and computer-readable media are disclosed for improved smart infrastructure data transfer. An example method may involve receiving, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol. The first information is converted from the first format into an agnostic format. An image, video, or real-time feed of an environment of the smart infrastructure device is captured. The first vehicle and a second vehicle in the image, video, or real-time feed is identified. It is determined that the second vehicle is temporarily or permanently incapable of performing a communication with the smart infrastructure device based on the image, video, or real-time feed. The image, video, or real-time feed is analyzed to generate second information associated with the second vehicle. The second information is converted into the agnostic format.

First claim

That which is claimed is:

1. A method comprising:
receiving, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;
converting the first information from the first format into an agnostic format;
capturing an image, video, or real-time feed of an environment of the smart infrastructure device;
identifying the first vehicle and a second vehicle in the image, video, or real-time feed;
determining that communication has not been received from the second vehicle;
responsive to determining that communication has not been received from the second vehicle, transmitting a communication request to the second vehicle;
determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device in response to not receiving a response from the second vehicle within a threshold period of time;
responsive to determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device, analyzing the image, video, or real-time feed to generate second information associated with the second vehicle;
converting the second information into the agnostic format;
converting the first information and the second information from the agnostic format into third information in the first format; and
sending the third information to the first vehicle.

2. The method of claim 1 , wherein the agnostic format includes a 3D coordinate map including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

3. The method of claim 1 , wherein determining that communication has not been received from the second vehicle comprises:
extracting information relating to a location of the first vehicle from the first information;
comparing the information relating to the location of the first vehicle that is extracted from the first information to a location of a vehicle in the image, video, or real-time feed;
determining that the location of the first vehicle that is extracted from the first information is a same location as the location of the vehicle in the image, video, or real-time feed;
analyzing the image, video, or real-time feed to identify a location of the second vehicle; and
determining that the location of the first vehicle from the first information is not the same as the location of the second vehicle in the image, video, or real-time feed.

4. The method of claim 1 , further comprising:
converting at least one of the first information or the second information into the third information in a second format associated with a second communication protocol; and
sending the third information in the second format to a third vehicle using the second communication protocol.

5. The method of claim 1 , further comprising:
receiving, by the smart infrastructure device and from a non-vehicle device, fourth information associated with the non-vehicle device in a fourth format associated with a fourth communication protocol; and
converting the fourth information into the agnostic format.

6. The method of claim 1 , wherein converting the first information from the first format into the agnostic format is performed using a look-up table including information about the first format.

7. A system comprising:
a processor; and
a memory storing computer-executable instructions, that when executed by the processor,
cause the processor to:
receive, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;
convert the first information from the first format into an agnostic format;
capture an image, video, or real-time feed of an environment of the smart infrastructure device;
identify the first vehicle and a second vehicle in the image, video, or real-time feed;
determine that communication has not been received from the second vehicle;
responsive to determining that communication has not been received from the second vehicle, transmit a communication request to the second vehicle;
determine that the second vehicle is permanently incapable of performing communication with the smart infrastructure device in response to not receiving a response from the second vehicle within a threshold period of time;
responsive to determining that the second vehicle is permanently incapable of performing communication with the smart infrastructure device, analyze the image, video, or real-time feed to generate second information associated with the second vehicle;
convert the second information into the agnostic format;
convert the first information and the second information from the agnostic format into third information in the first format; and
send the third information to the first vehicle.

8. The system of claim 7 , wherein the agnostic format includes a 3D coordinate map including a first coordinate representative of the first vehicle that is created based on the first information and a second coordinate representative of the second vehicle that is created based on the second information.

9. The system of claim 7 , wherein determining that communication has not been received from the second vehicle comprises:
extract information relating to a location of the first vehicle from the first information;
compare the information relating to the location of the first vehicle that is extracted from the first information to a location of a vehicle in the image, video, or real-time feed;
determine that the location of the first vehicle that is extracted from the first information is a same location as the location of the vehicle in the image, video, or real-time feed;
analyze the image, video, or real-time feed to identify a location of the second vehicle; and
determine that the location of the first vehicle from the first information is not the same as the location of the second vehicle in the image, video, or real-time feed.

10. The system of claim 7 , wherein the computer-executable instructions further cause the processor to:
convert at least one of the first information or the second information into the third information in a second format associated with a second communication protocol; and
send the third information in the second format to a third vehicle using the second communication protocol.

11. The system of claim 7 , wherein the computer-executable instructions further cause the processor to:
receive, by the smart infrastructure device and from a non-vehicle device, fourth information associated with the non-vehicle device in a fourth format associated with a fourth communication protocol; and
convert the fourth information into the agnostic format.

12. The system of claim 7 , wherein converting the first information from the first format into the agnostic format is performed using a look-up table including information about the first format.

13. A non-transitory computer readable medium including computer-executable instructions stored thereon, which when executed by one or more processors, cause the one or more processors to perform operations of:
receiving, by a smart infrastructure device and from a first vehicle, first information associated with the first vehicle in a first format associated with a first communication protocol;
converting the first information from the first format into an agnostic format;
capturing an image, video, or real-time feed of an enviro

Assignees

• Argo Ai Llc

Inventors

• Laverne Michel H J
• Biala Ilan

Classifications

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V20/54

  of traffic, e.g. cars on the road, trains or boats · CPC title

• G01C21/3423 Primary

  Multimodal routing · CPC title

• G01C21/3602

  Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera · CPC title

• G01C21/3893

  Transmission of map data from distributed sources, e.g. from roadside stations · CPC title