High-speed image readout and processing

Patent metadata

Field	Value
Publication number	US-2019208136-A1
Application number	US-201816214589-A
Country	US
Kind code	A1
Filing date	Dec 10, 2018
Priority date	Dec 29, 2017
Publication date	Jul 4, 2019
Grant date	—

Abstract

An optical system for a vehicle may be configured with a plurality of camera sensors. Each camera sensor may be configured to create respective image data of a respective field of view. The optical system is further configured with a plurality of image processing units coupled to the plurality of camera sensors. The image processing units are configured to compress the image data captured by the camera sensors. A computing system is configured to store the compressed image data in a memory. The computing system is further configured with a vehicle-control processor configured to control the vehicle based on the compressed image data. The optical system and the computing system can be communicatively coupled by a data bus.

First claim

What is claimed is:

1 ) An apparatus comprising:
an optical system configured with:
a plurality of camera sensors, wherein each camera sensor creates respective image data of a respective field of view of the respective camera sensor;
a plurality of image processing units coupled to the plurality of camera sensors, wherein the image processing units are configured to compress the image data captured by the camera sensors, and wherein the image processing units are located within an electrical distance of 6 inches of the camera sensors; and
a computing system configured with:
a memory configured to store the compressed image data;
a vehicle-control processor configured to control a vehicle based on the compressed image data,
a data bus configured to communicate the compressed image data between the optical system and the computing system.

2 ) The apparatus of claim 1 , wherein the data bus has a bandwidth that is greater than or equal to a bandwidth of the compressed image data, and wherein the data bus bandwidth is less than an bandwidth for the transmission of unprocessed image data.

3 ) The apparatus of claim 2 , wherein the plurality of camera sensors includes camera sensors arranged in eight sensor pairs, wherein the eight sensor pairs are arranged in a circular ring.

4 ) The apparatus of claim 3 , wherein the circular ring is configured to rotate.

5 ) The apparatus of claim 2 , wherein each camera sensor of the sensor pair is coupled to a different image processing unit than the other camera sensor of the sensor pair.

6 ) The apparatus of claim 1 , wherein the image processing unit is configured to compress a plurality of images by maintaining a first set of one or more images in the plurality of images and extracting motion data associated with a second set of one or more images in the plurality of images.

7 ) The apparatus of claim 1 , wherein the optical system is mounted in a sensor dome of the vehicle.

8 ) The apparatus of claim 1 , wherein the optical system is mounted behind a windshield of the vehicle.

9 ) A method comprising:
providing light to a plurality of camera sensors of an optical system to create image data corresponding to a respective field of view for each respective camera sensor; compressing the image data by a plurality of image processing units coupled to the plurality of camera sensors, and wherein the image processing units are located within an electrical distance of 6 inches of the camera sensors; communicating the compressed image data from the plurality of image processing units to a computing system; storing the compressed image data in a memory of the computing system; and controlling a vehicle based on the compressed image data by a vehicle-control processor of the computing system.

10 ) The method of claim 9 , further comprising capturing two images by a sensor pair comprising two camera sensors.

11 ) The method of claim 10 , wherein the images captured by each respective camera of the sensor pair are communicated to different image processing units, wherein compressing the image data by a plurality of image processing units comprises a different respective image processing unit compressing the image data from each camera sensor of the sensor pair.

12 ) The method of claim 11 , wherein the different image processing units are configured to process images received from the sensor pair simultaneously or near simultaneously.

13 ) The method of claim 8 , wherein compressing the image data comprises maintaining a first set of one or more images in the plurality of images and extracting motion data associated with a second set of one or more images in the plurality of images.

14 ) The method of claim 8 , wherein compressing the image data comprises storing a first image as a reference image and storing data related to changes with respect to the reference image for subsequent images, and storing a new reference image after a threshold is met.

15 ) A vehicle comprising:
a roof-mounted sensor unit comprising:
a first optical system configured with a first plurality of camera sensors, wherein each camera sensor of the first plurality of camera sensors creates respective image data of a respective field of view of the respective camera sensor,
a plurality of first image processing units coupled to the first plurality of camera sensors, wherein the first image processing units are configured to compress the image data captured by the first plurality of camera sensors,
a second camera unit comprising:
a second optical system configured with a second plurality of camera sensors, wherein each camera sensor of the second plurality of camera sensors creates respective image data of a respective field of view of the respective camera sensor,
a plurality of second image processing units coupled to the second plurality of camera sensors, wherein the second image processing units are configured to compress the image data captured by the second plurality of camera sensors,
a computing system located in the vehicle outside of the roof-mounted sensor unit, comprising:
a memory configured to store the compressed image data;
a control system configured to control the vehicle based on the compressed image data; and
a data bus configured to communicate the compressed image data between the roof-mounted sensor unit, the second camera unit, and the computing system.

16 ) The vehicle of claim 15 , wherein the first plurality of camera sensors includes camera sensors arranged in eight sensor pairs, wherein the eight sensor pairs are arranged in a circular ring.

17 ) The vehicle of claim 16 , wherein the circular ring is configured to rotate.

18 ) The vehicle of claim 16 , wherein each sensor pair includes a first camera sensor configure to image a scene with a first dynamic range corresponding to a first range of luminance levels and a second camera sensor configured to image the scene with a second dynamic range corresponding to a second range of luminance levels, wherein the second range of luminance levels includes luminance levels that are higher than the first range of luminance levels.

19 ) The vehicle of claim 16 , wherein each camera sensor of the sensor pair is coupled to a different image processing unit than the other camera sensor of the sensor pair.

20 ) The vehicle of claim 15 , wherein the image processing units are configured to compress images by maintaining a first set of one or more images in the plurality of images and extracting motion data associated with a second set of one or more images in the plurality of images.

Assignees

• Waymo Llc

Inventors

• Wendel Andreas
• Dittmer Jeremy
• Hermalyn Brendan

Classifications

• H04N7/12

  Systems in which the television signal is transmitted via one channel or a plurality of parallel channels, the bandwidth of each channel being less than the bandwidth of the television signal (H04N7/24 takes precedence) · CPC title

• B60R11/04

  Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle · CPC title

• B60R2011/0026

  Windows, e.g. windscreen · CPC title

• H04N5/917

  for bandwidth reduction · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title