Computer-assisted or autonomous driving with region-of-interest determination for traffic light analysis

Patent metadata

Field	Value
Publication number	US-10139832-B2
Application number	US-201715416893-A
Country	US
Kind code	B2
Filing date	Jan 26, 2017
Priority date	Jan 26, 2017
Publication date	Nov 27, 2018
Grant date	Nov 27, 2018

Abstract

Apparatuses, methods and storage medium associated with determining a ROI for traffic light analysis for CA/AD are disclosed herein. In embodiments, a system may include a region-of-interest (ROI) for traffic light analysis component to receive a current pose of the ego vehicle, and to apply the current pose of the ego vehicle to a plurality of traffic light poses to identify a ROI for a determination of a current state of a traffic light to be taken into consideration in provision of assistance to, or autonomous operation of the ego vehicle; and a traffic light analysis component to determine the current state of the traffic light within the ROI. Other embodiments may be disclosed or claimed.

First claim

What is claimed is:

1. A system for computer assisted or autonomous driving of an ego vehicle, comprising:
a region-of-interest for traffic light analysis (ROIFTLA) identification component to receive a current pose of the ego vehicle, and to apply the current pose of the ego vehicle to a plurality of traffic light poses to identify a region-of-interest (ROI) in an image of a frontal view of the ego vehicle, for a determination of a current state of a traffic light to be taken into consideration in provision of assistance to, or autonomous operation of, the ego vehicle, the traffic light being within the ROI in the image, and is one of a plurality of traffic lights in the frontal view of the ego vehicle, wherein each traffic light pose comprises coordinates and orientation relative to an origin of a traffic light coordinate system that is different from a map's coordinate system employed by the system to provide assistance to, or autonomously operate, the ego vehicle to traverse a route, and wherein the current pose of the ego vehicle comprises coordinates and yaw relative to an origin of the map's coordinate system, employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route; and
a traffic light analysis component coupled to the ROIFTLA identification component to determine the current state of the traffic light within the ROI in the image, and output the current state of the traffic light within the ROI in the image for use to provide assistance to, or autonomously operate, the ego vehicle, wherein to apply the current pose of the ego vehicle to the plurality of traffic light poses to identify the ROI in the image of a horizon for the determination of the current state of the traffic light within the ROI in the image to be taken into consideration, the ROIFTLA identification component is to transform the coordinates and yaw of the current pose of the ego vehicle from relative to the origin of the map's coordinate system to relative to the origin of the traffic light coordinate system, and apply a distance and an orientation constraint to the transformed coordinates and yaw of the current pose of the ego vehicle and the coordinates and orientations of the plurality of traffic light poses to identify the ROI in the image for the determination of the current state of the traffic light within the ROI in the image to be taken into consideration.

2. The system of claim 1 , wherein to determine the current state of the traffic light within the identified ROI in the images, the traffic light analysis component is to apply shape, contour or color analysis to a region of an image that corresponds to the identified ROI in the image.

3. The system of claim 2 , further comprising one or more sensors including a camera to capture the image.

4. The system of claim 1 , further comprising a localization component coupled to the ROIFTLA identification component to periodically determine in real time the current pose of the ego vehicle, and provide the current pose of the ego vehicle to the ROIFTLA identification component.

5. The system of claim 1 , further comprising a non-transitory computer readable storage medium coupled to the ROIFTLA identification component to store the plurality of traffic light poses.

6. The system of claim 1 , wherein the system is disposed in the ego vehicle.

7. A method for computerized assist or autonomous driving of an ego vehicle, comprising:
receiving, by a computerized assist or autonomous driving (CA/AD) system of the ego vehicle, a current pose of the ego vehicle, wherein the current pose of the ego vehicle comprises coordinates and yaw relative to an origin of a map's coordinate system, employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route;
applying, by the CA/AD system, the current pose of the ego vehicle to a plurality of traffic light poses to identify a region of interest (ROI) in an image of a front view of the ego vehicle as seen through a frontal view of the ego vehicle for determining a current state of a traffic light within the ROI in the image to be taken into consideration in providing assistance to, or autonomously operating the ego vehicle, the traffic light being one of a plurality of traffic lights in the front view of the ego vehicle, wherein each traffic light pose comprises coordinates and orientation relative to an origin of a traffic light coordinate system that is different from the map's coordinate system employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route, wherein to apply the current pose of the ego vehicle to the plurality of traffic light poses to identify the ROI in the image of a horizon for the determination of the current state of the traffic light within the ROI in the image of the horizon to be taken into consideration, an ROIFTLA identification component is to transform the coordinates and yaw of the current pose of the ego vehicle from relative to the origin of the map's coordinate system to relative to the origin of the traffic light coordinate system, and apply a distance and an orientation constraint to the transformed coordinates and yaw of the current pose of the ego vehicle and the coordinates and orientations of the plurality of traffic light poses to identify the ROI in the horizon for the determination of the current state of the traffic light within the ROI in the horizon to be taken into consideration;
determining, by the CA/AD system, the current state of the traffic light within the ROI in the image; and
providing, by the CA/AD system, assistance to, or autonomously operating the ego vehicle, based at least in part on the determined current state of the traffic light within the ROI in the image.

8. The method of claim 7 , wherein determining the current state of the traffic light within the identified ROI in the image comprises applying shape, contour or color analysis to a region of an image that corresponds to the identified ROI in image.

9. The method of claim 7 further comprising periodically determining, by the CA/AD system, in real time the current pose of the ego vehicle.

10. The method of claim 7 further comprising receiving and storing the plurality of traffic light poses by the CA/AD system.

11. At least one non-transitory computer readable media (CRM) comprising a plurality of instructions configured to cause a computerized assist or autonomous driving (CA/AD) system of an ego vehicle, in response to execution of the instructions by CA/AD system, to:
receive a current pose of the ego vehicle, wherein the current pose of the ego vehicle comprises coordinates and yaw relative to an origin of a map's coordinate system, employed by the system to provide assistance to, or autonomously operate the ego vehicle to traverse a route;
apply the current pose of the ego vehicle to a plurality of traffic light poses to identify a region of interest (ROI) in a image in front of the ego vehicle as seen through a frontal view of the ego vehicle for a determination of a current state of a traffic light within the ROI in the image to be taken into consideration in provision of assistance to, or autonomous operation of the ego vehicle, wherein each traffic light pose comprises coordinates and orientation relative to an origin of a traffic light coordinate system that is different from the map's coordinate system employed by the system to provide assistance to, or autonomously operate, the ego vehicle to traverse a route, wherein to apply the current pose of the ego vehicle to the plurality of traffic light poses to identify the ROI in the image of a horizon for the determination of the current s

Assignees

• Intel Corp

Inventors

• Sarkar Nandini
• Leow Teng Jun Nicholas

Classifications

• G08G1/09623 Primary

  Systems involving the acquisition of information from passive traffic signs by means mounted on the vehicle (G08G1/0967 takes precedence) · CPC title

• G05D1/0274 Primary

  using mapping information stored in a memory device (navigation using map-matching G01C21/30) · CPC title

• G05D1/0231

  using optical position detecting means (position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16) · CPC title

• G05D2201/0213

  Physics · mapped topic

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title