Road detection using traffic sign information

Patent metadata

Field	Value
Publication number	US-11113545-B2
Application number	US-201616345321-A
Country	US
Kind code	B2
Filing date	Dec 16, 2016
Priority date	Dec 16, 2016
Publication date	Sep 7, 2021
Grant date	Sep 7, 2021

Abstract

In some examples, one or more processors of a vehicle may store, in a computer-readable medium, sign information relating a sign type of a traffic sign to a distance to a road edge for a plurality of sign types. The one or more processors may receive at least one image from at least one sensor, and may recognize a sign type of a traffic sign in the at least one image. Further, the one or more processors may determine a likely location of a road edge in relation to the vehicle based at least partially on the recognized sign type and the stored sign information. In some examples, the one or more processors may send one or more control signals to at least one component of the vehicle based on the determined likely location of the road edge relative to the vehicle.

First claim

The invention claimed is:

1. A system comprising:
one or more processors; and
one or more non-transitory computer-readable media storing a database including sign information for a plurality of different sign types of traffic signs, the sign information including, for each sign type of the plurality of sign types, a predefined lateral distance from the sign type to a road edge, wherein the sign information in the database is initially generated at least using information specified by a jurisdictional entity, wherein the sign information in the database is subsequently updated using empirical data received from a plurality of vehicles over time;
the one or more non-transitory computer-readable media further including executable instructions, which, when executed by the one or more processors, configure the one or more processors to:
receive at least one image from at least one sensor;
recognize the sign type of a traffic sign in the at least one image; and
determine a likely location of the road edge in relation to a vehicle based at least partially on the recognized sign type and the predefined lateral distance specified by the sign information in the database, wherein determining the likely location of the road edge includes applying a first weight to the information specified by the jurisdictional entity in the database, and a second weight to the empirical data in the database.

2. The system as recited in claim 1 , wherein the one or more processors are further configured to send one or more control signals to at least one component of the vehicle based on the determined likely location of the road edge relative to the vehicle.

3. The system as recited in claim 2 , wherein the one or more control signals comprise a signal for controlling at least one of:
a steering component of the vehicle;
a braking component of the vehicle; or
a speed control component of an engine or motor driving the vehicle.

4. The system as recited in claim 1 , wherein the one or more processors are further configured to determine the likely location of the road edge by:
determining, based at least partially on the recognized sign type and the sign information, a target area of the at least one image where the road edge is likely to be located in the at least one image, wherein the target area is a subset of a total image area of the at least one image;
searching the target area of the at least one image for one or more features indicative of the road edge; and
determining the likely location of the road edge based at least partially on the one or more features.

5. The system as recited in claim 4 , wherein the one or more processors are further configured to:
determine the target area in a longitudinal direction corresponding to a direction of travel of the vehicle based on recognition of a first type of sign; and
determine the target area to include in the target area an area lateral to the direction of travel of the vehicle based on recognition of a second type of sign.

6. The system as recited in claim 1 , wherein:
the sign information in the database relates the sign type to the distance to the road edge for a plurality of sign types for a plurality of geographic jurisdictions; and
the one or more processors are further configured to:
receive geographic location information indicating a geographic location of the vehicle;
determine, from map information and the geographic location, a geographic jurisdiction in which the vehicle is indicated to be located; and
determine the likely location of the road edge in relation to the vehicle based at least partially on the recognized sign type, the sign information in the database, and indicated geographic jurisdiction.

7. The system as recited in claim 1 , wherein:
the database further includes road information indicating at least one of a distance from a road edge to a road centerline specified by the jurisdictional entity, or a size of travel lanes specified by the jurisdictional entity; and
the one or more processors are further configured to determine, based at least partially on the road information and the likely location of the road edge, at least one of:
a likely location of a centerline of a road on which the vehicle is located; or
a likely location of a travel lane of the road on which the vehicle is located.

8. The system as recited in claim 1 , wherein:
the database further includes road information indicating, for a plurality of types of roads for a plurality of geographic jurisdictions, at least one of a distance from a road edge to a road centerline specified by respective jurisdictional entities of respective geographic jurisdictions, or a size of travel lanes specified by respective jurisdictional entities of respective geographic jurisdictions; and
the one or more processors are further configured to:
receive geographic location information indicating a geographic location of the vehicle;
determine, from map information and the geographic location, a geographic jurisdiction in which the vehicle is indicated to be located; and
determine, based at least partially on the road information, the geographic jurisdiction, and the likely location of the road edge, at least one of:
a likely location of a centerline of a road on which the vehicle is located; or
a likely location of a travel lane of the road on which the vehicle is located.

9. The system as recited in claim 1 , wherein:
the sign information indicates, for individual sign types, the predefined lateral distance of the sign to the road edge and a predefined height of the sign from a road surface specified by the jurisdictional entity; and
the one or more processors are further configured to determine at least one of a likely road plane or a likely free space on the road based at least partially on the recognized sign type and the sign information.

10. The system as recited in claim 1 , further comprising at least one of a pair of cameras or a stereo camera as the at least one sensor,
wherein the one or more processors are further configured to determine the likely location of the road edge in relation to the vehicle based at least partially on a difference between a first position of the sign in a first image and second position of the sign in a second image, wherein the first image is taken by a first camera or through a first aperture of the stereo camera and the second image is taken by a second camera or through a second aperture of the stereo camera.

11. The system as recited in claim 1 , wherein the one or more processors are further configured to:
detect the road edge within a threshold level of confidence;
determine, from the at least one image, a distance from the road edge to the sign;
receive an indication of a geographic location of the vehicle; and
store, as empirical sign information, the determined distance from the road edge to the sign and the geographic location information.

12. The system as recited in claim 11 , wherein the one or more processors are further configured to:
send, over a network to a service computing device, the stored empirical sign information;
receive, from the service computing device, as a portion of the empirical data, updated sign information that has been updated based on empirical sign information sent to the service computing device from the plurality of vehicles; and
store the updated sign information in the database for use in determining a location of another road edge.

13. The system as recited in claim 1 , wherein the one or more processors are further configured to:
detect the road edge and one or more other road features within a threshold level of

Assignees

• Hitachi Automotive Systems Ltd

Inventors

• Kundu Subrata Kumar

Classifications

• G05D1/0231

  using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16) · CPC title

• G06V20/582 Primary

  of traffic signs · CPC title

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G06K9/00798 Primary

  Physics · mapped topic

• G06K9/00818

  Physics · mapped topic