Traffic light detection using multiple regions of interest and confidence scores

Patent metadata

Field	Value
Publication number	US-9977975-B2
Application number	US-201615240112-A
Country	US
Kind code	B2
Filing date	Aug 18, 2016
Priority date	Aug 18, 2016
Publication date	May 22, 2018
Grant date	May 22, 2018

Abstract

Described herein is a device for traffic light detection. The device comprises a memory and a traffic light detection module. The memory may store information, the information comprising first traffic light data of a first traffic light and second traffic light data of a second traffic light. The traffic light detection module may receive an image comprising a first candidate and a second candidate; determine a first region of interest based, at least in part, on the first traffic light data, the first region of interest comprising the first candidate; determine a second region of interest based, at least in part, on the second traffic light data, the second region of interest comprising the second candidate; and determine a confidence score for a first state of the first candidate, the confidence score based, at least in part, on a spatial relationship factor between the first candidate and the second candidate.

First claim

What is claimed is:

1. A device for traffic light detection comprising:
a memory configured to store information, the information comprising first traffic light data of a first traffic light and second traffic light data of a second traffic light; and
a traffic light detection module configured to:
receive an image comprising a first candidate and a second candidate;
receive the first traffic light data and the second traffic light data;
determine a first region of interest based, at least in part, on the first traffic light data, the first region of interest comprising the first candidate;
determine a second region of interest based, at least in part, on the second traffic light data, the second region of interest comprising the second candidate; and
determine a first confidence score for a first state of the first candidate, the first confidence score based, at least in part, on a first spatial relationship factor between the first candidate and the second candidate.

2. The device of claim 1 , wherein the traffic light detection module is further configured to:
compare the first confidence score to one or more additional confidence scores; and
determine a traffic light state for the first traffic light and the second traffic light, wherein the traffic light state corresponds to a greatest confidence score selected from the group consisting of the first confidence score and one of the one or more additional confidence scores.

3. The device of claim 1 , wherein the traffic light detection module is further configured to:
determine a first distance between the first candidate and the second candidate, the first distance based, at least in part, on the image; and
determine a second distance, the second distance based, at least in part, on the first traffic light data and the second traffic light data,
wherein the first spatial relationship factor is determined based, at least in part, on the first distance and the second distance.

4. The device of claim 1 , wherein the traffic light detection module is further configured to:
determine a first probability that the first candidate is in the first state, the first probability based, at least in part, an amount of time the first traffic light is in the first state, wherein the first confidence score is further based on the first probability.

5. The device of claim 1 , wherein the traffic light detection module is further configured to:
determine whether a first rule is satisfied;
increase the first confidence score if the first rule is satisfied; and
not increase the first confidence score if the first rule is not satisfied.

6. The device of claim 5 , wherein the first rule is that the first state is the same as a second state of the second candidate.

7. The device of claim 1 , wherein the traffic light detection module is further configured to:
determine a state transition probability of the first candidate, wherein the first confidence score is further based on the state transition probability.

8. The device of claim 1 , wherein the traffic light detection module is further configured to assign a first weighting parameter to the first spatial relationship factor.

9. The device of claim 1 , wherein the first state is selected from the group consisting of: red, green, yellow, and occluded.

10. A method for traffic light detection, the method comprising:
storing information comprising first traffic light data of a first traffic light and second traffic light data of a second traffic light;
receiving, from an image capture device, an image comprising a first candidate and a second candidate;
receiving the first traffic light data and the second traffic light data;
determining, by a traffic light detection module, a first region of interest based, at least in part, on the first traffic light data, the first region of interest comprising the first candidate;
determining, by the traffic light detection module, a second region of interest based, at least in part, on the second traffic light data, the second region of interest comprising the second candidate; and
determining, by the traffic light detection module, a first confidence score for a first state of the first candidate, the first confidence score based, at least in part, on a first spatial relationship factor between the first candidate and the second candidate.

11. The method of claim 10 further comprising:
comparing the first confidence score to one or more additional confidence scores; and
determining a traffic light state for the first traffic light and the second traffic light, wherein the traffic light state corresponds to a greatest confidence score selected from the group consisting of the first confidence score and one of the one or more additional confidence scores.

12. The method of claim 10 further comprising:
determining a first distance between the first candidate and the second candidate, the first distance based, at least in part, on the image; and
determining a second distance, the second distance based, at least in part, on the first traffic light data and the second traffic light data,
wherein the first spatial relationship factor is determined based, at least in part, on the first distance and the second distance.

13. The method of claim 10 further comprising:
determining a first probability that the first candidate is in the first state, the first probability based, at least in part, an amount of time the first traffic light is in the first state, wherein the first confidence score is further based on the first probability.

14. The method of claim 10 further comprising:
determining whether a first rule is satisfied;
increasing the first confidence score if the first rule is satisfied; and
not increasing the first confidence score if the first rule is not satisfied.

15. The method of claim 14 , wherein the first rule is that the first state is the same as a second state associated with the second candidate.

16. The method of claim 10 further comprising:
determining a state transition probability of the first candidate, wherein the first confidence score is further based on the state transition probability.

17. The method of claim 10 further comprising:
assigning a first weighting parameter to the first spatial relationship factor.

18. The method of claim 10 , wherein the first state is a member of the group consisting of:
red, green, yellow, and occluded.

19. The method of claim 18 further comprising:
determining a current state of the first traffic light and the second traffic light, based at least in part, on a highest confidence score of a plurality of confidence scores, wherein the first confidence score is a member of the plurality of confidence scores.

20. A method for traffic light detection, the method comprising:
receiving, from an image capture device, an image comprising a first candidate and a second candidate;
receiving first traffic light data of a first traffic light and second traffic light data of a second traffic light;
determining, by a traffic light detection module, a first region of interest based, at least in part, on the first traffic light data, the first region of interest comprising the first candidate;
determining, by the traffic light detection module, a second region of interest based, at least in part, on the second traffic light data, the second region of interest comprising the second candidate;
determining, by the traffic light detection module, a first spatial relationship factor between the

Assignees

• Toyota Eng & Mfg North America

Inventors

• Mei Xue
• Sakai Katsuhiro
• Prokhorov Danil V
• Delp Michael J

Classifications

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06K9/00825 Primary

  Physics · mapped topic

• G06K9/623

  Physics · mapped topic

• G06K9/6262

  Physics · mapped topic

• G06K9/2063

  Physics · mapped topic