Autonomous vehicle data recorders

Patent metadata

Field	Value
Publication number	US-11410475-B2
Application number	US-201916263403-A
Country	US
Kind code	B2
Filing date	Jan 31, 2019
Priority date	Jan 31, 2019
Publication date	Aug 9, 2022
Grant date	Aug 9, 2022

Abstract

Systems, methods and apparatus to collect sensor data generated in an autonomous vehicle. Sensors in the vehicle generate a sensor data stream during operations of the vehicle on a road. An advanced driver assistance system (ADAS) of the vehicle uses the sensor data stream to operate the vehicle and generate a trigger signal in response to a fault in object detection, recognition, identification or classification and/or in response to the detection/prediction of an accident. A cyclic buffer buffers at least a portion of the sensor data stream. In response to the trigger signal, a selected segment of the sensor data stream is stored into a non-volatile memory. The selected segment can be partially before the trigger signal and partially after the trigger signal; and selected segment can be longer than what can be fully buffered in the cyclic buffer at the time of the trigger signal.

First claim

What is claimed is:

1. An autonomous vehicle, comprising:
sensors configured to generate a sensor data stream during operations of the autonomous vehicle on a road;
an advanced driver assistance system configured to operate the autonomous vehicle on the road based on the sensor data stream, wherein the advanced driver assistance system is further configured to:
generate a training signal in response to a fault in object detection, recognition, identification or classification; and
generate an accident signal in response to detection or prediction of an accident; and

a data recorder having:
a communication interface coupled to the advanced driver assistance system;
a non-volatile memory;
a cyclic buffer configured to buffer the sensor data stream; and
a controller configured to control operations of buffering the sensor data stream into the cyclic buffer and copying content from the cyclic buffer into the non-volatile memory based on signals received via the communication interface from the advanced driver assistance system, wherein:
in absence of the training signal and the accident signal, the controller is configured to buffer continuously the sensor data stream into the cyclic buffer without copying data from the cyclic buffer into the non-volatile memory;
in response to the training signal, the controller is configured to store a first segment of the sensor data stream into the non-volatile memory by copying content from the cyclic buffer into the non-volatile memory while further buffering the sensor data stream into the cyclic buffer, wherein the first segment is longer than a second segment of the sensor data stream buffered in the cyclic buffer at receiving of the training signal; and
in response to the accident signal, the controller is configured to stop buffering data from the sensor data stream into the cyclic buffer.

2. The autonomous vehicle of claim 1 , wherein the first segment includes the second segment.

3. The autonomous vehicle of claim 2 , wherein the controller is configured to, as a response to the training signal, copy the second segment from the cyclic buffer to the non-volatile memory and write into the non-volatile memory a third segment that follows the second segment in the sensor data stream without the second segment going through any cyclic buffer.

4. The autonomous vehicle of claim 3 , wherein the controller is configured to, as a response to the training signal, copy the second segment to the non-volatile memory in parallel with writing the third segment into the non-volatile memory.

5. The autonomous vehicle of claim 2 , wherein the controller is configured to retrieve an oldest portion from the cyclic buffer while receiving a portion from the sensor data stream, and buffer the received portion into memory units occupied by the oldest portion.

6. The autonomous vehicle of claim 5 , wherein the controller is configured to, as a response to the training signal, write the oldest portion retrieved from the cyclic buffer in parallel with buffering the received portion into the cyclic buffer.

7. The autonomous vehicle of claim 1 , wherein the sensors include a digital camera, a radar, a lidar, or an ultrasound sonar, or any combination thereof.

8. The autonomous vehicle of claim 7 , wherein the advanced driver assistance system includes a map of the road and an artificial neural network; and the advanced driver assistance system is configured to generate the training signal in response to a mismatch between identification of an object in the map of the road and identification of a corresponding object via applying the sensor data stream in the artificial neural network.

9. The autonomous vehicle of claim 8 , wherein after the training signal, the autonomous vehicle is configured to transmit the first segment of the sensor data stream from the non-volatile memory to a remote server that is configured to use the first segment of the sensor data stream in at least updating the map of the road, the artificial neural network, or both the map and the artificial neural network.

10. The autonomous vehicle of claim 9 , wherein the controller is further configured to, in response to the accident signal, copy a content of the cyclic buffer into the non-volatile memory.

11. The autonomous vehicle of claim 1 , wherein the controller is further configured to identify a time period selection indication provided in the training signal and select the first segment according to the time period selection indication to copy the first segment into the non-volatile memory.

12. A data recorder of an autonomous vehicle, comprising:
a non-volatile memory;
a communication interface configured to receive a sensor data stream from sensors of the autonomous vehicle;
a first cyclic buffer configured to buffer the sensor data stream received in the communication interface; and
a controller configured to:
buffer, in absence of a trigger signal, the sensor data stream continuously into the first cyclic buffer;
in response to a training signal received in the communication interface as the trigger signal:
continue buffering the sensor data stream into the first cyclic buffer;
copy at least a part of a content of the first cyclic buffer into the non-volatile memory; and
store, into the non-volatile memory, a portion of the sensor data stream following the trigger signal without using a second cyclic buffer to buffer the portion of the sensor data stream; and

in response to an accident signal received in the communication interface as the trigger signal, stop buffering the sensor data stream into the first cyclic buffer.

13. The data recorder of claim 12 , wherein the training signal generated by an advanced driver assistance system of the vehicle is indicative of a fault in object detection, recognition, identification or classification.

14. The data recorder of claim 13 , wherein the controller is configured to receive a time window indicator and select the part and the portion according to the time window indicator.

15. The data recorder of claim 14 , wherein the controller is configured to copy the part and store the portion concurrently.

16. The data recorder of claim 15 , wherein the first cyclic buffer includes volatile memories; and the non-volatile memory includes NAND flash memory.

17. A method, comprising:
receiving, in a data recorder of an autonomous vehicle, a sensor data stream generated by sensors of the autonomous vehicle during operations of the autonomous vehicle on a road;
buffering the sensor data stream continuously into a cyclic buffer of the data recorder in absence of any trigger signal;
receiving, in the data recorder, a trigger signal generated by an advanced driver assistance system of the autonomous vehicle;
responsive to the trigger signal being a training signal:
continuing buffering the sensor data stream into the cyclic buffer;
copying a first segment of the sensor data stream from the cyclic buffer into a non-volatile memory of the data recorder;
storing, a second segment of the sensor data stream into the non-volatile memory, the second segment of the sensor data stream being generated after the trigger signal; and
transmitting, from the non-volatile memory to a remote server; and

responsive to the trigger signal being an accident signal, stopping buffering the sensor data stream into the cyclic buffer.

18. The method of claim 17 , further comprising:
detecting a mismatch between an object identification provided in a map of the road and an object identification made through applying the

Assignees

• Micron Technology Inc

Inventors

• Golov Gil

Classifications

• B60R21/0134 Primary

  responsive to imminent contact with an obstacle {, e.g. using radar systems} · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G07C5/085 Primary

  using electronic data carriers · CPC title

• B60R21/0136

  responsive to actual contact with an obstacle {, e.g. to vehicle deformation, bumper displacement or bumper velocity relative to the vehicle (monitoring crash strength in combination with passenger seat detection B60R21/01558)} · CPC title