Systems and methods for determining the lighting state of a vehicle

Patent metadata

Field	Value
Publication number	US-10061322-B1
Application number	US-201715480624-A
Country	US
Kind code	B1
Filing date	Apr 6, 2017
Priority date	Apr 6, 2017
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

Systems and method are provided for controlling a vehicle. In one embodiment, a vehicle lighting detection method includes receiving sensor data associated with operation of one or more vehicles, and extracting from the sensor data a plurality of images and a plurality of corresponding image labels, wherein the images each include at least a portion of an observed vehicle, and the image labels indicate the corresponding lighting state of the observed vehicle in each of the images. The method further includes training, with a processor, a machine learning model utilizing the plurality of images and the plurality of corresponding image labels.

First claim

What is claimed is:

1. A vehicle lighting detection method comprising:
receiving sensor data associated with operation of one or more vehicles;
extracting from the sensor data a plurality of images and a plurality of corresponding image labels, wherein the images each include at least a portion of an observed vehicle, and the image labels indicate a corresponding lighting state of the observed vehicle in each of the images;
training, with a processor, a machine learning model utilizing the plurality of images and the plurality of corresponding image labels.

2. The method of claim 1 , further including transmitting the trained machine learning model to an autonomous vehicle over a communication network.

3. The method of claim 1 , wherein the lighting states characterize the state of one or more conspicuity lights associated with the observed vehicle.

4. The method of claim 1 , wherein the plurality of images includes a sequence of images of the observed vehicle acquired at predetermined time intervals.

5. The method of claim 1 , wherein extracting the plurality of images from the sensor data includes cropping an optical image to include the portion of the vehicle based on the sensor data.

6. The method of claim 1 , wherein extracting the plurality of corresponding image labels is based on at least one of the state of the environment in the vicinity of the observed vehicle, the state of the observed vehicle relative to the environment, and a future behavior of the observed vehicle.

7. The method of claim 6 , wherein the state of the environment in the vicinity of the observed vehicle includes at least the state of a traffic light being approached by the observed vehicle.

8. The method of claim 6 , wherein the state of the observed vehicle relative to the environment includes at least a velocity of the observed vehicle, an acceleration of the observed vehicle, a type of the lane in which the observed vehicle is traveling, whether the observed vehicle is approaching an intersection, and whether the observed vehicle is approaching a light or signage that is likely to affect the motion of the vehicle.

9. The method of claim 6 , wherein the future behavior of the observed vehicle includes at least whether the observed vehicle turned and whether the observed vehicle stopped.

10. The method of claim 1 , wherein the machine learning model is an artificial neural network model.

11. The method of claim 10 , wherein the artificial neural network model is a convolutional neural network model.

12. A system for controlling an autonomous vehicle, comprising:
an image extraction module configured to:
accept sensor data associated with operation of one or more vehicles;
extract from the sensor data a plurality of images and a plurality of corresponding image labels, wherein the images each include at least a portion of an observed vehicle, and the image labels indicate the corresponding lighting state of the observed vehicle in each of the images; and
train a machine learning model utilizing the plurality of images and the plurality of corresponding image labels; and
a vehicle lighting detection module, including the trained machine learning model, configured to receive sensor data relating to an environment associated with the autonomous vehicle and determine the vehicle lighting state of a second vehicle in the environment.

13. The system of claim 12 , wherein the machine learning model is a convolutional neural network model.

14. The system of claim 12 , wherein the lighting states characterize the state of one or more conspicuity lights associated with the vehicle.

15. The system of claim 14 , wherein the conspicuity lights include at least one of brake lights, hazard lights, and turn-signal lights.

16. The system of claim 14 , wherein extracting the plurality of corresponding image labels is based on at least one of the state of the environment in the vicinity of the observed vehicle, the state of the observed vehicle relative to the environment, and the future behavior of the observed vehicle.

17. An autonomous vehicle, comprising:
at least one sensor that provides sensor data; and
a controller that, by a processor and based on the sensor data:
receives, over a network, an artificial neural network model trained utilizing a plurality of images and a plurality of corresponding image labels, wherein the images each include at least a portion of an observed vehicle, and the image labels indicate the corresponding lighting state of the observed vehicle in each of the images; and
determine, using the trained artificial neural network model, the vehicle lighting state of a second vehicle in the environment.

18. The autonomous vehicle of claim 17 , wherein the lighting states characterize the state of one or more conspicuity lights associated with the observed vehicle.

19. The autonomous vehicle of claim 17 , wherein the conspicuity lights include at least one of brake lights, hazard lights, and turn-signal lights.

20. The autonomous vehicle of claim 17 , wherein extracting the plurality of corresponding image labels is based on at least one of the state of the environment in the vicinity of the observed vehicle, the state of the observed vehicle relative to the environment, and the future behavior of the observed vehicle.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Palefsky-Smith Raphael

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F18/2414

  Smoothing the distance, e.g. radial basis function networks [RBFN] · CPC title

• G06N5/01

  Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title