Scalable video compression accelerator for autonomous driving

Patent metadata

Field	Value
Publication number	US-12323645-B2
Application number	US-202217906607-A
Country	US
Kind code	B2
Filing date	Aug 9, 2022
Priority date	Aug 9, 2022
Publication date	Jun 3, 2025
Grant date	Jun 3, 2025

Abstract

A video system includes a first data gathering node configured to receive a plurality of image streams from a plurality of cameras, respectively. Each of the plurality of cameras captures an environment of an autonomous driving vehicle (ADV). The first data gathering node tags the plurality of image streams with metadata that identifies each of the plurality of image streams, and combines the plurality of image streams with the metadata to form a combined image stream. A second data gathering node is communicatively coupled to the first data gathering node and is to receive the combined image stream from the first data gathering node and output the combined image stream with a second combined image stream.

First claim

What is claimed is:

1. A video system, comprising:
a first data gathering node configured to:
receive a first plurality of image streams from a first plurality of cameras, respectively, each of the first plurality of cameras capturing at least a portion of an environment of an autonomous driving vehicle (ADV),
tag the first plurality of image streams with metadata that identifies each of the first plurality of image streams, and
combine the first plurality of image streams with the metadata to form a first combined image stream; and
a second data gathering node configured to:
receive the first combined image stream from the first data gathering node, and
output the first combined image stream with a second combined image stream obtained from a second plurality of cameras;
wherein the second plurality of cameras are different from the first plurality of cameras.

2. The video system of claim 1 , wherein the second data gathering node is further configured to:
receive a second plurality of image streams from the second plurality of cameras, respectively;
tag the second plurality of image streams with second metadata that identifies each of the second plurality of image streams;
combine the second plurality of image streams with the second metadata to form the second combined image stream; and
output the second combined image stream and the first combined image stream to a third data gathering node or to a data processing node.

3. The video system of claim 1 , wherein the first data gathering node receives the first plurality of image streams from the first plurality of cameras through a respective image buffer.

4. The video system of claim 1 , wherein the first data gathering node includes a plurality of channel links, wherein a first of the plurality of channel links that carries the first combined image stream and wherein each remaining one of the plurality of channel links is not used.

5. The video system of claim 1 , wherein the second data gathering node includes a second plurality of channel links, wherein a first of the second plurality of channel links receives and carries the first combined image stream and a second of the second plurality of channel links carries the second combined image stream, and each remaining one of the second plurality of channel links are not used.

6. The video system of claim 1 , wherein the first data gathering node is further configured to buffer the first combined image stream.

7. The video system of claim 1 , further comprising a first data processing node, configured to:
receive at least the first combined image stream and the second combined image stream;
extract each of the first plurality of image streams from the first combined image stream based on the metadata;
compress the first plurality of image streams with a plurality of processing engines of the first data processing node, resulting in a first plurality of compressed image streams; and
output at least the second combined image stream to a second data processing node.

8. The video system of claim 7 , wherein the second data processing node is configured to:
receive at least the second combined image stream;
extract each of a second plurality of image streams from the second combined image stream based on the second metadata;
compress the second plurality of image streams with a second plurality of processing engines of the second data processing node, resulting in a second plurality of compressed image streams; and
output an additional combined image stream to a third data processing node in response to receiving the additional combined image stream from the first data processing node.

9. The video system of claim 8 , wherein the first data processing node is further configured to buffer the first combined image stream prior to extracting each of the first plurality of image streams from the first combined image stream.

10. The video system of claim 8 , wherein the first plurality of compressed image streams and the second plurality of compressed image streams are stored in non-volatile computer-readable memory.

11. An autonomous driving vehicle (ADV), comprising:
a first plurality of cameras; and
a video compression system comprising
a first data gathering node configured to:
receive a first plurality of image streams from the first plurality of cameras, respectively, each of the first plurality of cameras capturing at least a portion of an environment of the autonomous driving vehicle (ADV),
tag the first plurality of image streams with metadata that identifies each of the first plurality of image streams, and
combine the first plurality of image streams with the metadata to form a first combined image stream; and
a second data gathering node configured to:
receive the first combined image stream from the first data gathering node, and
output the first combined image stream with a second combined image stream obtained from a second plurality of cameras;
wherein the second plurality of cameras are different from the first plurality of cameras.

12. The ADV of claim 11 , wherein the second data gathering node is further configured to:
receive a second plurality of image streams from the second plurality of cameras, respectively;
tag the second plurality of image streams with second metadata that identifies each of the second plurality of image streams;
combine the second plurality of image streams with the second metadata to form the second combined image stream; and
output the second combined image stream and the first combined image stream to a third data gathering node or to a data processing node.

13. The ADV of claim 11 , wherein the first data gathering node receives the first plurality of image streams from the first plurality of cameras through a respective image buffer.

14. The ADV of claim 11 , wherein the first data gathering node includes a plurality of channel links, wherein a first of the plurality of channel links that carries the first combined image stream and wherein each remaining one of the plurality of channel links is not used.

15. The ADV of claim 11 , wherein the second data gathering node includes a second plurality of channel links, wherein a first of the second plurality of channel links receives and carries the first combined image stream and a second of the second plurality of channel links carries the second combined image stream, and each remaining one of the second plurality of channel links are not used.

16. The ADV of claim 11 , wherein the first data gathering node is further configured to buffer the first combined image stream.

17. The ADV of claim 11 , wherein the video compression system further comprise a first data processing node, configured to:
receive at least the first combined image stream and the second combined image stream;

extract each of the first plurality of image streams from the first combined image stream based on the metadata;
compress the first plurality of image streams with a plurality of processing engines of the first data processing node, resulting in a first plurality of compressed image streams; and

output at least the second combined image stream to a second data processing node.

18. The ADV of claim 17 , wherein the second data processing node is configured to:
receive at least the second combined image stream; extract each of a second plurality of image streams from the second combined image stream based on the second metadata;
compress the second plurality of image streams with a second plurality of processing engines of t

Assignees

• Baidu Usa Llc
• Apollo Intelligent Driving Technology Beijing Co Ltd

Inventors

• Wang Qiang
• Zhang Manjiang
• Wang Shuai

Classifications

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• H04N21/84

  Generation or processing of descriptive data, e.g. content descriptors {(systems specially adapted for using meta-information in broadcast systems H04H60/73)} · CPC title

• H04N21/43632

  involving a wired protocol, e.g. IEEE 1394 (high-speed IEEE 1394 serial bus H04L12/40052) · CPC title

• H04N21/41422

  located in transportation means, e.g. personal vehicle (arrangements specially adapted for transportation systems in broadcast systems H04H20/62) · CPC title

• H04N21/4334

  Recording operations (recording of a television signal H04N5/76; arrangements for recording or accumulating broadcast information or broadcast-related information H04H60/27) · CPC title