Systems and methods for controlling autonomous vehicle in real time

Patent metadata

Field	Value
Publication number	US-10933884-B2
Application number	US-202016903401-A
Country	US
Kind code	B2
Filing date	Jun 17, 2020
Priority date	Jun 17, 2019
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

Systems and methods for operating a vehicle automatically in real time may be provided. The system may receive and process real time sensor data from a sensing system via an arithmetic and control unit (ACU). The system may obtain, via the ACU, one or more tasks during the processing of the real time sensor data. The system may classify, via the ACU, the one or more tasks into real time vehicle controlling (VC) tasks and non real time VC tasks. The system may further send the real time VC tasks to at least one dedicated processing core of the ACU for processing the real time VC tasks and generating one or more real time VC commands accordingly.

First claim

We claim:

1. A system for operating a vehicle automatically in real time, the system comprising:
at least one storage device storing at least one set of instructions;
an arithmetic and control unit (ACU), a real-time transmission unit, and a vehicle controlling unit (VCU) in communication with the at least one storage device to execute the at least one set of instructions,
wherein the ACU comprises a multiple core processor having at least one processing core dedicated for processing one or more real time vehicle controlling (VC) tasks and the ACU is configured to receive and process real time sensor data from a sensing system to generate one or more real time VC commands, wherein the ACU is further configured to obtain and classify one or more tasks into the one or more real time VC tasks and tasks that are not real time VC tasks (non real time VC tasks),
wherein the real-time transmission unit is configured for transmitting the one or more real time VC commands to the VCU in real time; and
wherein the VCU is configured for receiving and executing the one or more real time VC commands generated by the ACU to operate the vehicle in real time accordingly,
wherein the ACU is operated by a real time operating system including at least one kernel for assigning the one or more real time VC tasks to the at least one dedicated processing core to generate the one or more real time VC commands.

2. The system of claim 1 , wherein the real time sensor data includes signals from at least one of a LiDAR, a radar, a camera, or a positioning system.

3. The system of claim 1 , wherein the real time operating system is a real time Linux system.

4. The system of claim 1 , wherein the real-time transmission unit is configured to be operated independently from the ACU.

5. The system of claim 1 , wherein the real-time transmission unit is configured to control the vehicle when the ACU is disabled or irresponsive.

6. The system of claim 1 , wherein the one or more real time VC commands includes at least one of acceleration, deceleration, making a turn, or switching lanes.

7. The system of claim 1 , wherein the real time sensor data is updated continuously by the system and the ACU generates updated real time VC commands based on updated real time sensor data, which is in turn transmitted to the VCU via the real-time transmission unit, and there is a time interval between the transmission of a real time VC command and its immediate updated real time VC command that is measured against an interval threshold, and the system further comprising a time limit with a set of accompany instructions stored in the storage device to instruct the system to:
determine whether the time interval is within the interval threshold;
if the time interval is shorter than or equal to the interval threshold, send, via the real-time transmission unit, the updated real time VC commands to VCU to control the vehicle accordingly;
if the time interval is longer than the interval threshold, determine whether the difference between the time interval and the interval threshold is within a time limit;
if the difference is longer than the time limit, generate, by the real-time transmission unit, a real time VC command to directly command VCU to slow down the vehicle, and
alternatively, if the difference is shorter than or equal to the time limit, repeat or maintain the real time VC command.

8. The system of claim 7 , wherein the set of accompany instructions further instructs the system to alert a user of the vehicle.

9. A method for operating a vehicle automatically in real time, comprising:
receiving and processing, via an arithmetic and control unit (ACU), real time sensor data from a sensing system;
obtaining, via the ACU, one or more tasks during the processing of the real time sensor data;
classifying, via the ACU, the one or more tasks into real time vehicle controlling (VC) tasks and non real time VC tasks; and
sending the real time VC tasks to at least one dedicated processing core of the ACU for processing the real time VC tasks and generating one or more real time VC commands accordingly,
wherein the ACU is operated by a real time operating system including at least one kernel for assigning the one or more real time VC tasks to the at least one dedicated processing core to generate the one or more real time VC commands.

10. The method of claim 9 , wherein the method further comprising:
sending, via a real-time transmission unit, the one or more real time VC commands to a vehicle controlling unit (VCU) in real time for controlling the vehicle.

11. The method of claim 9 , wherein the real time sensor data include signals from at least one of a LiDAR, a radar, a camera, or a global positioning system (GPS).

12. The method of claim 10 , wherein the real-time transmission unit includes at least one processor that is independent from the ACU and is configured to be operated independently from the ACU to directly control the vehicle when the ACU is disabled or irresponsive.

13. The method of claim 9 , wherein the real time operating system is a real time Linux system.

14. The method of claim 9 , wherein the one or more real time VC commands includes at least one of acceleration, deceleration, making a turn, or switching lanes.

15. The method of claim 10 , wherein the real time sensor data is updated continuously by the system and the ACU generates updated real time VC commands based on the updated real time sensor data, which is in turn transmitted to the VCU via the real-time transmission unit, and there is a time interval between the transmission of a real time VC command and its immediate updated real time VC command that is measured against an interval threshold, the method further comprising:
determining whether the time interval is within the interval threshold;
if the time interval is shorter than or equal to the interval threshold, sending, via the real-time transmission unit, the updated real time VC commands to VCU to control the vehicle accordingly;
if the time interval is longer than the interval threshold, determining whether the difference between the time interval and the interval threshold is within a time limit;
if the difference is longer than the time limit, generating, by the real-time transmission unit, a real time VC command to directly command VCU to slow down the vehicle, and
alternatively, if the difference is shorter than or equal to the time limit, repeating or maintaining the real time VC command.

16. The method of claim 9 further comprising:
sending an alert signal for a user of the vehicle.

17. A non-transitory computer readable medium, comprising at least one set of instructions, wherein when executed by at least one processor of a computing device, the at least one set of instructions causes the computing device to perform a method, the method comprising:
receiving and processing, via an arithmetic and control unit (ACU), real time sensor data from a sensing system;
obtaining, via the ACU, one or more tasks during the processing of the real time sensor data;
classifying, via the ACU, the one or more tasks into real time vehicle controlling (VC) tasks and non real time VC tasks; and
sending the real time VC tasks to at least one dedicated processing core of the ACU for processing the real time VC tasks and generating one or more real time VC commands accordingly,
wherein the ACU is operated by a real time operating system including at least one kernel for assigning the one or more real time VC tasks to the at least one dedicated processing core to generate the one or

Assignees

• Beijing Didi Infinity Technology & Dev Co Ltd

Inventors

• Wang Fenglei
• Wang Jintao

Classifications

• G06F9/4887

  involving deadlines, e.g. rate based, periodic · CPC title

• B60W10/20

  including control of steering systems · CPC title

• B60W2050/046

  involving external transmission of data to or from the vehicle, e.g. via telemetry, satellite, Global Positioning System [GPS] · CPC title

• B60W2050/0006

  Digital architecture hierarchy · CPC title

• B60W10/04

  including control of propulsion units · CPC title