Early detection of turning condition identification using perception technology

Patent metadata

Field	Value
Publication number	US-9959624-B2
Application number	US-201614996628-A
Country	US
Kind code	B2
Filing date	Jan 15, 2016
Priority date	Dec 22, 2014
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

An in-vehicle system for identifying turn-only and shared turn lanes and identifying turn rules according to visual indicia captured from turn signs. An image apparatus configured to obtain image data including graphic data and/or text data from a roadway sign, as well as street light image data for an intersection on the roadway. The image data may be processed to detect a graphic image and/or text from the image data relating to the roadway sign. Street light image data may also be captured and processed to determine a street light status. An identification module may determine rules for turning on the intersection of the roadway based on the processed image data relating to the roadway sign, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the determined street light status.

First claim

The invention claimed is:

1. An in-vehicle system for determining rules for a turn on a roadway, the system comprising:
a first image apparatus for obtaining lane marking image data from the roadway;
a lane marking detection module for detecting lane markings on the roadway from the lane marking image data;
a second image apparatus for obtaining at least one of image data and text data from a roadway sign;
a sign analyzer module for detecting at least one of a graphic image and text from the at least one of image data and text data respectively;
means for detecting lane markings as one of standard frequency dashed lane markings, high frequency dashed lane markings, and solid lane markings and for identifying a turn-only lane and a shared turn lane in response to the detected lane markings;
and means for determining rules for turning for an intersection of the roadway based on the at least one of the graphic image and recognition and interpretation of the detected text.

2. The in-vehicle system of claim 1 , wherein the second image apparatus is configured to obtain street light image data for the intersection.

3. The in-vehicle system of claim 2 , wherein the means for determining rules for turning for an intersection is configured to compare the street light image data with the determined rules.

4. The in-vehicle system of claim 3 , further comprising communications for transmitting a warning signal if the means for determining rules for turning for an intersection determines that the street light image data does not comply with the determined rules.

5. The in-vehicle system of claim 3 , further comprising communications for transmitting a turn engagement signal if the means for determining rules for turning for an intersection determines that the street light image data complies with the determined rules.

6. An in-vehicle system for determining rules for a turn on a roadway, the system comprising:
an image apparatus configured to obtain image data comprising (i) at least one of graphic data and text data from a roadway sign, and (ii) street light image data for an intersection on the roadway;
an analyzer module configured to process the image data to detect at least one of a graphic image and text from the image data relating to the roadway sign, and to process the street light image data to determine a street light status; and
an identification module for determining rules for turning on the intersection of the roadway based on the processed image data relating to the roadway sign, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the determined street light status, and
wherein the identification module is further configured to determine the rules for turning on the intersection based at least in part on recognition and interpretation of the detected text data.

7. The in-vehicle system of claim 6 , wherein the analyzer module is configured to perform optical character recognition (OCR) from the image data relating to the roadway sign to detect text data.

8. The in-vehicle system of claim 7 , wherein the identification module is configured to determine the rules for turning on the intersection based at least in part on the detected text data.

9. The in-vehicle system of claim 8 , further comprising communications for transmitting a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

10. The in-vehicle system of claim 6 , wherein the analyzer module is configured to detect graphic images from the graphic data.

11. The in-vehicle system of claim 9 , wherein the identification module is configured to compare the detected graphic image to one or more stored graphic images to determine a matching graphic image, wherein the identification module is configured to determine the rules for turning on the intersection based at least in part on the matching graphic image.

12. The in-vehicle system of claim 11 , further comprising communications for transmitting a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

13. The in-vehicle system of claim 6 , further comprising:
a second image apparatus for obtaining lane marking image data from the roadway; and
a lane marking detection module for detecting lane markings on the roadway from the lane marking image data,
wherein the lane marking detection module is configured to detect lane markings as one of standard frequency dashed lane markings, high frequency dashed lane markings, and solid lane markings and for identifying a turn-only lane and a shared turn lane in response to the detected lane markings.

14. The in-vehicle system of claim 13 , wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the detected lane marking.

15. A method for determining rules for a turn on a roadway via an in-vehicle system, comprising:
obtaining image data via an image apparatus, the image data comprising (i) at least one of graphic data and text data from a roadway sign, and (ii) street light image data for an intersection on the roadway;
processing the image data via an analyzer module to detect at least one of a graphic image and text from the image data relating to the roadway sign, and to process the street light image data to determine a street light status; and
determining rules for turning on the intersection of the roadway, via an identification module, based on the processed image data relating to the roadway sign, wherein the identification module is further configured to determine if the turn on the roadway is allowed based on the determined street light status;
wherein the identification module is further configured to determine the rules for turning on the intersection based at least in part on recognition and interpretation of the detected text data.

16. The method of claim 15 , further comprising performing optical character recognition (OCR) from the image data relating to the roadway sign via the analyzer module to detect text data.

17. The method of claim 16 , further comprising determining the rules for turning on the intersection, via the identification module, based at least in part on the detected text data.

18. The method of claim 17 , further comprising transmitting, via communications, a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

19. The method of claim 15 , further comprising detecting, via the analyzer module, graphic images from the graphic data.

20. The method of claim 19 , further comprising comparing, via the identification module, the detected graphic image to one or more stored graphic images to determine a matching graphic image, and determining the rules for turning on the intersection based at least in part on the matching graphic image.

21. The method of claim 20 , further comprising transmitting, via communications, a warning signal if the identification module determines that the determined street light status does not comply with the determined rules.

22. The method of claim 15 , further comprising:
obtaining, via a second image apparatus, lane marking image data from the roadway; and
detecting, via a lane marking detection module, lane markings on the roadway from the lane marking image data,
wherein detecting lane

Assignees

• Volkswagen Ag
• Audi Ag

Inventors

• Raghu Kaushik
• Beatrix Schmidts

Classifications

• G06T7/004 Primary

  Physics · mapped topic

• B60R1/00

  Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles · CPC title

• G06K9/00798

  Physics · mapped topic

• B60R2300/308

  by overlaying the real scene, e.g. through a head-up display on the windscreen · CPC title

• G06K9/00818

  Physics · mapped topic