System and method for detecting taillight signals of a vehicle

Patent metadata

Field	Value
Publication number	US-10387736-B2
Application number	US-201715709832-A
Country	US
Kind code	B2
Filing date	Sep 20, 2017
Priority date	Sep 20, 2017
Publication date	Aug 20, 2019
Grant date	Aug 20, 2019

Abstract

A system method for detecting taillight signals of a vehicle using a convolutional neural network is disclosed. A particular embodiment includes: receiving a plurality of images from one or more image-generating devices; generating a frame for each of the plurality of images; generating a ground truth, wherein the ground truth includes a labeled image with one of the following taillight status conditions for a right or left taillight signal of the vehicle: (1) an invisible right or left taillight signal, (2) a visible but not illuminated right or left taillight signal, and (3) a visible and illuminated right or left taillight signal; creating a first dataset including the labeled images corresponding to the plurality of images, the labeled images including one or more of the taillight status conditions of the right or left taillight signal; and creating a second dataset including at least one pair of portions of the plurality of images, wherein the at least one pair of portions of the plurality of the images are in temporal succession.

First claim

What is claimed is:

1. A method of detecting a taillight signal of a vehicle, the method comprising:
receiving a plurality of images from one or more image-generating devices;
generating a frame for each of the plurality of images;
generating a ground truth, wherein the ground truth includes a labeled image with one of the following taillight status conditions for a right or left taillight signal of the vehicle: (1) an invisible right or left taillight signal, (2) a visible but not illuminated right or left taillight signal, and (3) a visible and illuminated right or left taillight signal;
creating a first dataset including the labeled images corresponding to the plurality of images, the labeled images including one or more of the taillight status conditions of the right or left taillight signal; and
creating a second dataset including at least one pair of portions of the plurality of images, wherein the at least one pair of portions of the plurality of the images are in temporal succession.

2. The method of the claim 1 further including generating a loss function by combining the first and the second dataset.

3. The method of the claim 1 further including determining a confidence level using parameters.

4. The method of the claim 1 further including outputting parameters to a temporal fusion process.

5. The method of the claim 1 further including adjusting parameters of a temporal fusion process.

6. The method of the claim 1 wherein generating the ground truth further includes labelling the plurality of data by adding annotations manually.

7. The method of the claim 1 further including determining a length of time of a cycle of taillight signal on and taillight signal off.

8. The method of the claim 1 wherein the taillight signal includes a turn signal, a brake signal, and an emergency stop signal.

9. The method of the claim 1 further including providing statistics of temporal patterns of taillight signals to a temporal fusion process.

10. A system comprising:
a data processor; and
a taillight signal recognition processing module, executable by the data processor, the taillight signal recognition processing module being configured to perform a taillight signal recognition operation using a convolutional neural network, the taillight signal recognition operation being configured to:
receive a plurality of images from one or more image-generating devices;
generate a frame for each of the plurality of images;
generate a ground truth, wherein the ground truth includes a labeled image with one of the following taillight status conditions for a right or left taillight signal of the vehicle: (1) an invisible right or left taillight signal, (2) a visible but not illuminated right or left taillight signal, and (3) a visible and illuminated right or left taillight signal;
create a first dataset including the labeled images corresponding to the plurality of images, the labeled images including one or more of the taillight status conditions of the right or left taillight signal; and
create a second dataset including at least one pair of portions of the plurality of images, wherein the at least one pair of portions of the plurality of the images are in temporal succession.

11. The system of the claim 10 being further configured to generate a loss function by combining the first and the second dataset.

12. The system of the claim 10 being further configured to determine a confidence level using parameters.

13. The system of the claim 10 being further configured to output parameters to a temporal fusion process.

14. The system of the claim 10 being further configured to adjust parameters of a temporal fusion process.

15. The system of the claim 10 being further configured to generate the ground truth by labelling the plurality of data by adding annotations manually.

16. The system of the claim 10 being further configured to determine a length of time of a cycle of taillight signal on and taillight signal off.

17. The system of the claim 10 wherein the taillight signal includes a turn signal, a brake signal, and an emergency stop signal.

18. The system of the claim 10 being further configured to provide statistics of temporal patterns of taillight signals to a temporal fusion process.

Assignees

• TuSimple

Inventors

• Wang Yijie
• Zhu Ligeng
• Wang Panqu
• Chen Pengfei

Classifications

• G06V10/774 Primary

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V20/584 Primary

  of vehicle lights or traffic lights · CPC title

• G06F18/25

  Fusion techniques · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title