Vision-based indicator signal detection using spatiotemporal filtering

Patent metadata

Field	Value
Publication number	US-9892327-B2
Application number	US-201615331079-A
Country	US
Kind code	B2
Filing date	Oct 21, 2016
Priority date	Jun 26, 2013
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

An autonomous vehicle is configured to detect an active turn signal indicator on another vehicle. An image-capture device of the autonomous vehicle captures an image of a field of view of the autonomous vehicle. The autonomous vehicle captures the image with a short exposure to emphasize objects having brightness above a threshold. Additionally, a bounding area for a second vehicle located within the image is determined. The autonomous vehicle identifies a group of pixels within the bounding area based on a first color of the group of pixels. The autonomous vehicle also calculates an oscillation of an intensity of the group of pixels. Based on the oscillation of the intensity, the autonomous vehicle determines a likelihood that the second vehicle has a first active turn signal. Additionally, the autonomous vehicle is controlled based at least on the likelihood that the second vehicle has a first active turn signal.

First claim

What is claimed is:

1. A method comprising:
receiving, from an image-capture device coupled to a first vehicle, an image of a field of view of the first vehicle, wherein the image-capture device is configured to capture an image with a short exposure to emphasize objects having a brightness above a threshold;
determining a first bounding area within the image including a depiction of a second vehicle;
determining a second bounding area within the image including a depiction of a third vehicle, wherein at least a portion of the first bounding box overlaps the second bounding box;
identifying a group of pixels in the image based on an intensity of the group of pixels, wherein the group of pixels is located within the overlapping portion of the bounding boxes;
determining, based on an oscillation of the intensity of the group of pixels, a likelihood that the group of pixels is an active indicator signal;
determining which of the second vehicle and the third vehicle the group of pixels is associated with based on the size of the group of pixels; and
providing instructions to control, using a computing device, the first vehicle based at least on the likelihood that the determined vehicle has an active indicator signal.

2. The method of claim 1 , wherein the bounding area is determined based on a previously stored image.

3. The method of claim 1 , wherein the oscillation is between 0.75 and 2.5 Hertz.

4. The method of claim 1 , further comprising:
determining, based on a second oscillation of a second group of pixels, a likelihood that the determined vehicle has another active indicator signal; and
determining, based on both the likelihood that the determined vehicle has an active indicator signal and the likelihood that the determined vehicle has another active indicator signal, a likelihood that the second vehicle has active emergency lights.

5. The method of claim 1 , wherein the third vehicle obscures the view of at least a portion of the second vehicle.

6. The method of claim 1 , further comprising:
identifying a group of pixels in the image based on a second color of the group of pixels, wherein the group of pixels is located within a bounding area of the determined vehicle;
determining a correlation between an intensity of a first color of the group of pixels and an intensity of the second color of the group of pixels; and
determining, based on the correlation, a likelihood that the determined vehicle has the active indicator signal.

7. The method of claim 6 , further comprising identifying a sun glare based on the correlation.

8. A control system comprising:
at least one processor; and
receiving, from an image-capture device coupled to a first vehicle, an image of a field of view of the first vehicle, wherein the image-capture device is configured to capture an image with a short exposure to emphasize objects having a brightness above a threshold;
determining a first bounding area within the image including a depiction of a second vehicle;
determining a second bounding area within the image including a depiction of a third vehicle, wherein at least a portion of the first bounding box overlaps the second bounding box;
identifying a group of pixels in the image based on an intensity of the group of pixels, wherein the group of pixels is located within the overlapping portion of the bounding boxes;
determining, based on an oscillation of the intensity of the group of pixels, a likelihood that the group of pixels is an active indicator signal;
determining which of the second vehicle and the third vehicle the group of pixels is associated with based on the size of the group of pixels; and
providing instructions to control, using a computing device, the first vehicle based at least on the likelihood that the determined vehicle has an active indicator signal.

9. The control system of claim 8 , wherein the bounding area is determined based on a previously stored image.

10. The control system of claim 8 , wherein the oscillation is between 0.75 and 2.5 Hertz.

11. The control system of claim 8 , further comprising:
determining, based on a second oscillation of a second group of pixels, a likelihood that the determined vehicle has another active indicator signal; and
determining, based on both the likelihood that the determined vehicle has an active indicator signal and the likelihood that the second vehicle has another active indicator signal, a likelihood that the second vehicle has active emergency lights.

12. The control system of claim 8 , wherein the third vehicle obscures the view of at least a portion of the second vehicle.

13. The control system of claim 8 , further comprising:
identifying a group of pixels in the image based on a second color of the group of pixels, wherein the group of pixels is located within the bounding area of the determined vehicle;
determining a correlation between an intensity of a first color of the group of pixels and an intensity of the second color of the group of pixels; and
determining, based on the correlation, a likelihood that the determined vehicle has the first active turn signal.

14. A non-transitory computer readable medium having stored thereon instructions, that when executed by a computing device, cause the computing device to perform functions comprising:
receiving, from an image-capture device coupled to a first vehicle, an image of a field of view of the first vehicle, wherein the image-capture device is configured to capture an image with a short exposure to emphasize objects having a brightness above a threshold;
determining a first bounding area within the image including a depiction of a second vehicle;
determining a second bounding area within the image including a depiction of a third vehicle, wherein at least a portion of the first bounding box overlaps the second bounding box;
identifying a group of pixels in the image based on an intensity of the group of pixels, wherein the group of pixels is located within the overlapping portion of the bounding boxes;
determining, based on an oscillation of the intensity of the group of pixels, a likelihood that the group of pixels is an active indicator signal;
determining which of the second vehicle and the third vehicle the group of pixels is associated with based on the size of the group of pixels; and
providing instructions to control, using a computing device, the first vehicle based at least on the likelihood that the determined vehicle has an active indicator signal.

15. The non-transitory computer readable medium of claim 14 , wherein the bounding area is determined based a previously stored image.

16. The non-transitory computer readable medium of claim 14 , wherein the oscillation is between 0.75 and 2.5 Hertz.

17. The non-transitory computer readable medium of claim 14 , further comprising:
determining, based on a second oscillation of a second group of pixels, a likelihood that the determined vehicle has a second active turn signal; and
determining, based on both the likelihood that the determined vehicle has a first active turn signal and the likelihood that the determined vehicle has a second active turn signal, a likelihood that the second vehicle has active emergency lights.

18. The non-transitory computer readable medium of claim 14 , wherein the third vehicle obscures the view of at least a portion of the second vehicle.

19. The non-transitory computer readable medium of claim 14 , further comprising:
identifying a group of pixels in the image based on a second color of the group of

Assignees

• Waymo Llc

Inventors

• Ogale Abhijit
• Ferguson Dave

Classifications

• G06V20/584 Primary

  of vehicle lights or traffic lights · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• H04N23/73

  by influencing the exposure time · CPC title

• B60R2300/80

  characterised by the intended use of the viewing arrangement · CPC title

• H04N5/2353

  Electricity · mapped topic