Brake light detection

Patent metadata

Field	Value
Publication number	US-12067764-B2
Application number	US-202318321392-A
Country	US
Kind code	B2
Filing date	May 22, 2023
Priority date	Nov 22, 2016
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

Systems, methods, and devices for detecting brake lights are disclosed herein. A system includes a mode component, a vehicle region component, and a classification component. The mode component is configured to select a night mode or day mode based on a pixel brightness in an image frame. The vehicle region component is configured to detect a region corresponding to a vehicle based on data from a range sensor when in a night mode or based on camera image data when in the day mode. The classification component is configured to classify a brake light of the vehicle as on or off based on image data in the region corresponding to the vehicle.

First claim

What is claimed is:

1. A method comprising:
clustering input range data based on density-based spatial clustering to generate a three-dimensional clustered object;
receiving a classification from a neural network indicating whether the three-dimensional clustered object likely depicts a vehicle;
corresponding the three-dimensional clustered object from the input range data with a region of interest of an input image; and
determining whether the region of interest depicts a brake light based on data from the input image.

2. The method of claim 1 , further comprising determining whether dimensions for the clustered object are within a size range corresponding with the vehicle; and
providing the clustered object to the neural network in response to determining the dimensions for the clustered object are within the size range corresponding with the vehicle.

3. The method of claim 1 , further comprising removing a ground plane from the input range data, and wherein the clustering the input range data occurs after removing the ground plane.

4. The method of claim 1 , wherein the input range data comprises light detection and ranging (LIDAR) data captured by a LIDAR system of a parent vehicle;
wherein the input image comprises image sensor data captured by a camera of the parent vehicle; and
wherein the input range data and the input image are captured substantially simultaneously.

5. The method of claim 1 , wherein the neural network is a deep neural network comprising a deep architecture trained to output a binary classification indicating whether the three-dimensional clustered object depicts the vehicle or does not depict the vehicle.

6. The method of claim 1 , wherein corresponding the three-dimensional clustered object from the input range data with the region of interest of the input image comprises corresponding based on pixel locations for the region of interest of the input image.

7. The method of claim 1 , further comprising converting pixel data for the region of interest of the input image to lightness and two-color channels (LAB) color space.

8. The method of claim 7 , wherein determining whether the region of interest comprises the brake light comprises:
extracting contours from the pixel data for the region of interest that is converted to the LAB color space;
based on a shape and size of extracted contours, determining whether any brake light region exists within the pixel data for the region of interest.

9. The method of claim 8 , further comprising, in response to identifying a brake light region of interest within the pixel data, providing the brake light region of interest to a neural network trained on a dataset of brake lights to classify the brake light as on or off.

10. The method of claim 9 , further comprising receiving an output from the neural network indicating whether the brake light depicted within the region of interest is likely on or off.

11. The method of claim 1 , further comprising:
calculating an average pixel brightness of a plurality of pixels in the input image;
determining whether the average pixel brightness exceeds or falls below a threshold value; and
in response to the average pixel brightness falling below the threshold value, converting the input image to LAB color space.

12. The method of claim 1 , further comprising:
feeding the region of interest to a neural network trained on a dataset comprising a plurality of brake lights;
receiving as output from the neural network an indication of whether the region of interest comprises a positive brake light that is turned on or a negative brake light that is not turned on.

13. The method of claim 1 , wherein each of the input range data and the input image is captured by a camera of a parent vehicle, and wherein the method further comprises providing instructions to an automated driving/assistance system of the parent vehicle to execute a driving maneuver in response to determining whether the region of interest depicts the brake light.

14. The method of claim 1 , wherein the input image is captured by a camera of a parent vehicle, and wherein the method further comprises:
determining whether the brake light is on or off in response to determining the region of interest comprises the brake light; and
in response to determining the brake light is on, providing a notification to a driver of the parent vehicle and/or an automated driving/assistance system of the parent vehicle;
wherein the notification comprises an indication that the vehicle in front of the parent vehicle is braking.

15. A non-transitory computer readable storage medium storing instructions for execution by one or more processors, the instructions comprising:
clustering input range data based on density-based spatial clustering to generate a three-dimensional clustered object;
receiving a classification from a neural network indicating whether the three-dimensional clustered object likely depicts a vehicle;
corresponding the three-dimensional clustered object from the input range data with a region of interest of an input image; and
determining whether the region of interest depicts a brake light based on data from the input image.

16. The non-transitory computer readable storage medium of claim 15 , wherein the instructions further comprise determining whether dimensions for the clustered object are within a size range corresponding with the vehicle; and
providing the clustered object to the neural network in response to determining the dimensions for the clustered object are within the size range corresponding with the vehicle.

17. The non-transitory computer readable storage medium of claim 15 , wherein the neural network is a deep neural network comprising a deep architecture trained to output a binary classification indicating whether the three-dimensional clustered object depicts the vehicle or does not depict the vehicle.

18. The non-transitory computer readable storage medium of claim 15 , wherein the instructions are such that corresponding the three-dimensional clustered object from the input range data with the region of interest of the input image comprises corresponding based on pixel locations for the region of interest of the input image.

19. The non-transitory computer readable storage medium of claim 15 , wherein the instructions further comprise converting pixel data for the region of interest of the input image to lightness and two-color channels (LAB) color space.

20. The non-transitory computer readable storage medium of claim 19 , wherein the instructions are such that determining whether the region of interest comprises the brake light comprises:
extracting contours from the pixel data for the region of interest that is converted to the LAB color space;
based on a shape and size of extracted contours, determining whether any brake light region exists within the pixel data for the region of interest.

Assignees

• Ford Global Tech Llc

Inventors

• Moosaei Maryam
• Hotson Guy
• Mahmoudieh Parsa
• Nariyambut Murali Vidya

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V20/584

  of vehicle lights or traffic lights · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title