Multiplexing sensor data

Patent metadata

Field	Value
Publication number	US-11529916-B1
Application number	US-201916728567-A
Country	US
Kind code	B1
Filing date	Dec 27, 2019
Priority date	Dec 27, 2019
Publication date	Dec 20, 2022
Grant date	Dec 20, 2022

Abstract

The described techniques relate to a simulation system that multiplexes sensor data from multiple sensors and outputs the multiplexed sensor data in channels corresponding to the multiple sensors to appropriate vehicle systems at an appropriate time based on encoded timestamp data. In examples, a multiplexer may receive sensor datasets associated with different sensors. The multiplexer may encode the different sensor datasets with timestamp data and supplemental data to generate an encoded dataset. The multiplexer may output the encoded dataset to a video output port to transmit the encoded dataset to a demultiplexer. The demultiplexer receives the encoded dataset from the video output port, and separates the encoded dataset into channels corresponding to the sensors from which the sensor data was received. The demultiplexer may output the datasets in the respective channels at a time (or times) indicated in the timestamp data and according to the supplemental data.

First claim

What is claimed is:

1. A system comprising:
a multiplexer configured to:
receive a first data set associated with a first sensor and a second data set associated with a second sensor; and
encode the first data set and the second data set with timestamp data and supplemental data to generate an encoded data set, wherein the supplemental data further comprises at least one of a frame type, a frame size, or a fault injection parameter configured to simulate a sensor fault;

a video output port communicatively coupled to the multiplexer to transmit the encoded data set from the multiplexer; and
a demultiplexer communicatively coupled to the video output port and configured to:
receive the encoded data set from the video output port;
separate the encoded data set into a first channel of data comprising the first data set and a second channel of data comprising the second data set; and
output at least one of the first channel of data or the second channel of data to a vehicle system indicated in the supplemental data and at a time indicated by the timestamp data.

2. The system of claim 1 , wherein encoding the first data set and the second data set is based at least in part on a timing determination, the timing determination being included in the timestamp data.

3. The system of claim 1 , wherein:
the first data set comprises sensor data generated by or simulated by the first sensor, wherein the first sensor comprises a camera, a lidar sensor, a radar sensor, a long-wave infrared sensor, or a time-of-flight sensor; and
the second data set comprises sensor data generated by or simulated by a different one of the camera, the lidar sensor, the radar sensor, the long-wave infrared sensor, or the time-of-flight sensor.

4. The system of claim 1 , wherein the demultiplexer comprises a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

5. The system of claim 1 , wherein the multiplexer is further configured to combine the first data set and the second data set into an ultra-high-resolution image frame.

6. A method comprising:
receiving, at a multiplexer, a first data set associated with a first sensor and a second data set associated with a second sensor;
encoding the first data set and the second data set with timestamp data and supplemental data to generate an encoded data set, wherein the supplemental data further comprises at least one of a frame type, a frame size, or a fault injection parameter configured to simulate a sensor fault; and
outputting the encoded data set to a demultiplexer.

7. The method of claim 6 , wherein outputting the encoded data set comprises outputting the encoded data via a display port, an HDMI port, a serial port, an ethernet port, or a wireless network.

8. The method of claim 6 , further comprising:
receiving, at the demultiplexer, the encoded data set;
separating the encoded data set into a first channel of data comprising the first data set and a second channel of data comprising the second data set; and
outputting at least one of the first channel of data or the second channel of data to a vehicle system indicated in the supplemental data and at a time indicated by the timestamp data.

9. The method of claim 6 , wherein:
the first data set comprises sensor data generated by or simulated by the first sensor, wherein the first sensor comprises a camera, a lidar sensor, a radar sensor, a long-wave infrared sensor, or a time-of-flight sensor; and
the second data set comprises sensor data generated by or simulated by a different one of the camera, the lidar sensor, the radar sensor, the long-wave infrared sensor, or the time-of-flight sensor.

10. The method of claim 6 , wherein the demultiplexer comprises a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

11. The method of claim 6 , wherein the multiplexer is further configured to combine the first data set and the second data set into an ultra-high-resolution image frame.

12. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving, at a multiplexer, a first data set associated with a first sensor and a second data set associated with a second sensor;
encoding the first data set and the second data set with timestamp data and supplemental data to generate an encoded data set, wherein the supplemental data further comprises at least one of a frame type, a frame size, or a fault injection parameter configured to simulate a sensor fault; and
outputting the encoded data set to a demultiplexer.

13. The one or more non-transitory computer-readable media of claim 12 , wherein outputting the encoded data comprises outputting the encoded data set via a display port, an HDMI port, a serial port, an ethernet port, or a wireless network.

14. The one or more non-transitory computer-readable media of claim 12 , the operations further comprising:
receiving, at the demultiplexer, the encoded data set;
separating the encoded data set into a first channel of data comprising the first data set and a second channel of data comprising the second data set; and
outputting at least one of the first channel of data or the second channel of data to a vehicle system indicated in the supplemental data and at a time indicated by the timestamp data.

15. The one or more non-transitory computer-readable media of claim 12 , wherein:
the first data set comprises sensor data generated by or simulated by the first sensor, wherein the first sensor comprises a camera, a lidar sensor, a radar sensor, a long-wave infrared sensor, or a time-of-flight sensor; and
the second data set comprises sensor data generated by or simulated by a different one of the camera, the lidar sensor, the radar sensor, the long-wave infrared sensor, or the time-of-flight sensor.

16. The one or more non-transitory computer-readable media of claim 12 , wherein the demultiplexer comprises a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC).

17. The one or more non-transitory computer-readable media of claim 12 , wherein the multiplexer is further configured to combine the first data set and the second data set into an ultra-high-resolution image frame.

18. The one or more non-transitory computer-readable media of claim 12 , wherein encoding the first data set and the second data set is based at least in part on a timing determination, the timing determination being included in the timestamp data.

19. The method of claim 6 , wherein encoding the first data set and the second data set is based at least in part on a timing determination, the timing determination being included in the timestamp data.

20. The system of claim 1 , wherein the multiplexer is further communicatively coupled to at least one of a display port, an HDMI port, a serial port, an ethernet port, or a wireless network to transmit the encoded data set from the multiplexer to the demultiplexer.

Assignees

• Zoox Inc

Inventors

• Karadeniz Turhan
• Khanoyan Garen
• Mcmichael Ryan

Classifications

• H04N21/41422

  located in transportation means, e.g. personal vehicle (arrangements specially adapted for transportation systems in broadcast systems H04H20/62) · CPC title

• H04N21/23614

  Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28) · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• G08C15/06 Primary

  successively, i.e. using time division · CPC title