Vision-based object detection using a polar grid

Patent metadata

Field	Value
Publication number	US-9766628-B1
Application number	US-201414244988-A
Country	US
Kind code	B1
Filing date	Apr 4, 2014
Priority date	Apr 4, 2014
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

A computing device of a first vehicle may receive a first image and a second image of a second vehicle having flashing light signals. The computing device may determine, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle. The computing device may determine a polar grid that partitions the image region in the first image and the second image into polar bins, and identify portions of image data exhibiting a change in color and a change in brightness between the first image and the second image. The computing device may determine a type of the flashing light signals and a type of the second vehicle; and accordingly provide instructions to control the first vehicle.

First claim

What is claimed is:

1. A method comprising:
receiving, at a computing device of a first vehicle, a first image of a second vehicle captured by an image-capture device coupled to the first vehicle and a second image captured by the image-capture device subsequent to capturing the first image, wherein the second vehicle has one or more flashing light signals;
determining, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle;
determining a polar grid that partitions the image region in the first image and the second image into a plurality of polar bins, wherein each polar bin of the plurality of polar bins is defined by two lines extending from a center portion of the image region to about a boundary of the image region;
identifying, based on a comparison of image content of polar bins in the first image to image content of corresponding polar bins in the second image, one or more portions of image data exhibiting a change in color and a change in brightness between the first image and the second image;
determining a type of the one or more flashing light signals of the second vehicle and a type of the second vehicle based on (i) a number of portions of image data exhibiting the change in color and the change in brightness, (ii) the color of the one or more portions, and (iii) the brightness of the one or more portions; and
providing, by the computing device, instructions to control the first vehicle based on the type of the second vehicle and the type of the one or more flashing light signals.

2. The method of claim 1 , wherein identifying the one or more portions of image data comprises:
within each polar bin in the first image, identifying a number of image portions that exhibit characteristics of a bright light having an intensity greater than a threshold intensity;
within each corresponding polar bin in the second image, identifying corresponding image portions; and
determining the change in color and the change in brightness for the image portions from the polar bin in the first image to the corresponding polar bin in the second image.

3. The method of claim 1 , wherein the polar bins are non-uniform in size.

4. The method of claim 1 , further comprising:
determining that several polar bins of the plurality of polar bins are void of image pixels exhibiting the change in color or the change in brightness between the first image and the second image; and
merging the several polar bins into a single polar bin.

5. The method of claim 1 , wherein the second vehicle is moving relative to the first vehicle such that the image region that bounds the second vehicle in the first image is different from a respective image region that bounds the second vehicle in the second image.

6. The method of claim 1 , further comprising:
assigning each pixel in the image region to a respective polar bin of the polar bins.

7. The method of claim 1 , wherein determining the type of the second vehicle comprises determining that the second vehicle is an emergency vehicle characterized by multiple flashing light signals.

8. The method of claim 1 , wherein determining the type of a given flashing light signal of the one or more flashing light signals comprises determining that the given flashing light is a vehicular turn signal or a brake light.

9. The method of claim 1 , wherein the first image and the second image are captured consecutively.

10. The method of claim 1 , wherein the computing device is configured to control the first vehicle in an autonomous operation mode.

11. A non-transitory computer readable medium having stored thereon executable instructions that, upon execution by a computing device of a first vehicle, cause the computing device to perform functions comprising:
receiving a first image of a second vehicle captured by an image-capture device coupled to the first vehicle and a second image captured by the image-capture device subsequent to capturing the first image, wherein the second vehicle has one or more flashing light signals;
determining, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle;
determining a polar grid that partitions the image region in the first image and the second image into a plurality of polar bins, wherein each polar bin of the plurality of polar bins is defined by two lines extending from a center portion of the image region to about a boundary of the image region;
identifying, based on a comparison of image content of polar bins in the first image to image content of corresponding polar bins in the second image, one or more portions of image data exhibiting a change in color and a change in brightness between the first image and the second image;
determining a type of the one or more flashing light signals of the second vehicle and a type of the second vehicle based on (i) a number of portions of image data exhibiting the change in color and the change in brightness, (ii) the color of the one or more portions, and (iii) the brightness of the one or more portions; and
providing instructions to control the first vehicle based on the type of the second vehicle and the type of the one or more flashing light signals.

12. The non-transitory computer readable medium of claim 11 , wherein the function of identifying the one or more portions of image data comprises:
within each polar bin in the first image, identifying a number of image portions that exhibit characteristics of a bright light having an intensity greater than a threshold intensity;
within each corresponding polar bin in the second image, identifying corresponding image portions; and
determining the change in color and the change in brightness for the image portions from the polar bin in the first image to the corresponding polar bin in the second image.

13. The non-transitory computer readable medium of claim 11 , wherein the polar bins are non-uniform in size.

14. The non-transitory computer readable medium of claim 11 , wherein the functions further comprise:
determining that several polar bins of the plurality of polar bins are void of image pixels exhibiting the change in color or the change in brightness between the first image and the second image; and
merging the several polar bins into a single polar bin.

15. The non-transitory computer readable medium of claim 11 , wherein the functions further comprise:
assigning each pixel in the image region to a respective polar bin of the polar bins.

16. A system comprising:
an image-capture device coupled to a first vehicle;
at least one processor in communication with the image-capture device; and
a memory having stored thereon executable instructions that, upon execution by the at least one processor, cause the system to perform functions comprising:
receiving a first image of a second vehicle captured by the image-capture device and a second image captured by the image-capture device subsequent to capturing the first image, wherein the second vehicle has one or more flashing light signals;
determining, in the first image and the second image, an image region that bounds the second vehicle such that the image region substantially encompasses the second vehicle;
determining a polar grid that partitions the image region in the first image and the second image into a plurality of polar bins, wherein each polar bin of the plurality of polar bins is defined by two lines extending from a center portion of the image region to abou

Assignees

• Waymo Llc

Inventors

• Lo Wan-Yen
• Ferguson David Ian Franklin
• Ogale Abhijit

Classifications

• G06V20/584 Primary

  of vehicle lights or traffic lights · CPC title

• G06V10/421

  by analysing segments intersecting the pattern · CPC title

• G06K9/4652

  Physics · mapped topic

• G05D2201/0212

  Physics · mapped topic

• G06K9/00825

  Physics · mapped topic