Automated labeling of traffic control features

Patent metadata

Field	Value
Publication number	US-2025118095-A1
Application number	US-202318482156-A
Country	US
Kind code	A1
Filing date	Oct 6, 2023
Priority date	Oct 6, 2023
Publication date	Apr 10, 2025
Grant date	—

Abstract

Sensor data obtained from vehicles driving through a particular environment (e.g., a particular city or region being mapped) is used to identify and label traffic control features. Location, speed, and/or acceleration of mapping vehicles can be used to identify intersections that may have traffic control features. Environmental data, such as image data captured by one or more cameras, and point cloud data collected by lidar and/or radar sensors, is used to automatically detect traffic control features at the intersections.

First claim

What is claimed is:

1 . A computer implemented method for identifying a traffic control feature, the method comprising:
identifying, in a two-dimensional image obtained at a particular location on a roadway, a group of points; projecting the group of points onto a three-dimensional point cloud obtained at the particular location on the roadway, wherein each point of the group of points has a respective depth relative to the particular location on the roadway; identifying a subgroup of points of the group of points based on the depths of the points in the group of points; determining that the subgroup of points corresponds to a particular traffic control feature; and determining a location of the particular traffic control feature along the roadway based on a position in three-dimensional space of the subgroup of points.

2 . The method of claim 1 , wherein the subgroup of points is a first subgroup of points, and determining that the subgroup of points corresponds to the particular traffic control feature comprises:
identifying, in a second two-dimensional image obtained at a second location on the roadway, a second group of points; projecting the second group of points onto a second three-dimensional point cloud obtained at the second location on the roadway; identifying a second subgroup of points based on the depths of the points in the second group of points; and determining that the second subgroup of points substantially corresponds to the first subgroup of points.

3 . The method of claim 2 , further comprising:
calculating a first confidence that the first subgroup of points corresponds to the particular traffic control feature; calculating a second confidence that the second subgroup of points corresponds to the particular traffic control feature; and determining that the second confidence exceeds a threshold confidence.

4 . The method of claim 1 , wherein the two-dimensional image and the point cloud are obtained from a vehicle traveling along the roadway when the vehicle is at the particular location.

5 . The method of claim 4 , wherein the vehicle comprises a camera to capture the two-dimensional image and a light detection and ranging (LIDAR) sensor to capture the point cloud.

6 . The method of claim 1 , wherein the depths of the points in the identified subgroup of points represent a substantially flat surface.

7 . The method of claim 1 , further comprising selecting the particular location on the roadway to identify the particular traffic control feature, the particular location on the roadway proximate to an intersection, and the two-dimensional image and the three-dimensional point cloud representing at least a portion of the intersection.

8 . The method of claim 7 , further comprising:
retrieving data representing pathways driven by a plurality of vehicles; and identifying the intersection based on the data representing pathways, wherein at least two pathways represented in the data cross at the intersection.

9 . The method of claim 1 , wherein the particular traffic control feature is a stop sign, wherein determining that the subgroup of points corresponds to the particular traffic control feature comprises:
accessing data describing vehicle behavior of a vehicle approaching the stop sign; and determining that the vehicle behavior indicates that the vehicle stopped at a position proximate to the stop sign.

10 . The method of claim 1 , wherein the particular traffic control feature is a traffic light, the method further comprising:
determining a boundary of the traffic light; and identifying at least one bulb within the boundary of the traffic light.

11 . A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:
identify, in a two-dimensional image obtained at a particular location on a roadway, a group of points; project the group of points onto a three-dimensional point cloud obtained at the particular location on the roadway, wherein each point of the group of points has a respective depth relative to the particular location on the roadway; identify a subgroup of points of the group of points based on the depths of the points in the group of points; determine that the subgroup of points corresponds to a particular traffic control feature; and determine a location of the particular traffic control feature along the roadway based on a position in three-dimensional space of the subgroup of points.

12 . The computer-readable medium of claim 11 , wherein the subgroup of points is a first subgroup of points, and determining that the subgroup of points corresponds to the particular traffic control feature comprises:
identifying, in a second two-dimensional image obtained at a second location on the roadway, a second group of points; projecting the second group of points onto a second three-dimensional point cloud obtained at the second location on the roadway; identifying a second subgroup of points based on the depths of the points in the second group of points; and determining that the second subgroup of points substantially corresponds to the first subgroup of points.

13 . The computer-readable medium of claim 11 , wherein the two-dimensional image and the point cloud are obtained from a vehicle traveling along the roadway when the vehicle is at the particular location, the vehicle comprising a camera to capture the two-dimensional image and a light detection and ranging (LIDAR) sensor to capture the point cloud.

14 . The computer-readable medium of claim 11 , wherein the depths of the points in the identified subgroup of points represent a substantially flat surface.

15 . The computer-readable medium of claim 11 , wherein the instructions further cause the processor to select the particular location on the roadway to identify the particular traffic control feature, the particular location on the roadway proximate to an intersection, and the two-dimensional image and the three-dimensional point cloud representing at least a portion of the intersection.

16 . A system comprising:
a database storing image data and point cloud data obtained by a vehicle traveling along a roadway; and computer circuitry to:
identify, in the image data, a group of points;
project the group of points onto point cloud data obtained at a same position on the roadway as the image data, wherein each point of the group of points has a respective depth relative to the position on the roadway;
identify a subgroup of points of the group of points based on the depths of the points in the group of points;
determine that the subgroup of points corresponds to a particular traffic control feature; and
determine a location of the particular traffic control feature along the roadway based on a position in three-dimensional space of the subgroup of points.

17 . The system of claim 16 , wherein the subgroup of points is a first subgroup of points, the computer circuitry further to:
identify, in a second image obtained at a second location on the roadway, a second group of points; project the second group of points onto a second point cloud obtained at the second location on the roadway; identify a second subgroup of points based on the depths of the points in the second group of points; and determine that the second subgroup of points substantially corresponds to the first subgroup of points.

18 . The system of claim 16 , wherein the vehicle comprises a camera to capture image data and a light detection and ranging (LIDAR) sens

Assignees

• Gm Cruise Holdings Llc

Inventors

• Lin Chia-Wei
• Fasola Juan
• Kushwaha Shreyans
• Ahmed Jabeer

Classifications

• G01C21/3815

  Road data · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06V20/584

  of vehicle lights or traffic lights · CPC title

• G06V20/70 Primary

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G08G1/01

  Detecting movement of traffic to be counted or controlled (G08G1/07 - G08G1/14 take precedence) · CPC title