System and method for determining vehicle data set familiarity

Patent metadata

Field	Value
Publication number	US-11087144-B2
Application number	US-201816156357-A
Country	US
Kind code	B2
Filing date	Oct 10, 2018
Priority date	Oct 10, 2018
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

The present disclosure relates to systems, devices and methods for identifying objects and scenarios that have not been trained or are unidentifiable to vehicle perception sensors or vehicle assistive driving systems. Embodiments are directed to using a trained vehicle data set to identify target objects in vehicle sensor data. In one embodiment, a process is provided that includes running a scene detection operation on vehicle to derive a vector of target object attributes of the vehicle sensor data and generating a vector representation for the scene detection operation and the attributes of the vehicle sensor data. The vector representation compared to a familiarity vector to represent effectiveness of the scene detection operation. In addition, the vector representation can be scored to identify one or more target objects or significant scenarios, including unidentifiable objects and/or driving scenes, scenarios for reporting.

First claim

What is claimed is:

1. A method for identifying significant scenario data by a control unit of a vehicle, the method comprising:
receiving, by a control unit, vehicle sensor data captured by at least one sensor of the vehicle, the vehicle sensor data generated by at least one perception sensor of a vehicle;
running, by the control unit, a scene detection operation on the vehicle sensor data to derive a vector of target object attributes of the vehicle sensor data;
comparing, by the control unit, the vector of target object attributes against a cluster of a vector data model of a trained vehicle data set; and
identifying, by the control unit, significant scenario data based on a divergence between the vector of target object attributes and the cluster, wherein the significant scenario data identifies at least one target object of the vehicle sensor data.

2. The method of claim 1 , wherein the vehicle sensor data includes at least one of image, radar, and LiDAR data for a detection zone of a driver assistance system of the vehicle.

3. The method of claim 1 , wherein running the scene detection operation on the vehicle sensor data generates an annotated data set for target objects in real time based on the attributes of the trained vehicle data set, the trained vehicle data set providing a plurality of object types and object attributes.

4. The method of claim 1 , wherein the cluster of the vector data model of the trained vehicle data set is generated by a k-means clustering algorithm.

5. The method of claim 1 , wherein identifying the significant scenario includes determining that a target object not represented in the trained vehicle data set.

6. The method of claim 1 , wherein identifying the significant scenario includes determining that the trained vehicle data set is unable to classify a target object.

7. The method of claim 1 , wherein identifying the significant scenario includes determining familiarity of a target object relative to the trained data set based on at least one of a number of target objects, classification of target objects, size and shape of target object, object type, and object color.

8. The method of claim 1 , wherein identifying the significant scenario includes determining at least one vehicle operation characteristic as an attribute relative to identification of a target object in at least one of a driver assistance system and autonomous driving system.

9. The method of claim 1 , wherein identifying the significant scenario includes determining familiarity for a current scene of vehicle operation relative to a vehicle trained data set for at least one driving condition.

10. The method of claim 1 , further comprising scoring, by the control unit, the vector on an ability of the scene detection operation to perceive target object attributes of the vehicle sensor data using the trained vehicle data set, and wherein the significant scenario is identified based on a score of the vector of target object attributes being below a predetermined threshold.

11. The method of claim 1 , further comprising outputting the at least one target object of the vehicle sensor data.

12. A vehicle control unit comprising:
an input configured to receive vehicle sensor data; and
a control unit coupled to the input, wherein the control unit is configured to:
receive vehicle sensor data captured by at least one sensor of the vehicle, the vehicle sensor data generated by at least one perception sensor of a vehicle,
run a scene detection operation on the vehicle sensor data to derive a vector of target object attributes of the vehicle sensor data,
compare a vector representation for the scene detection operation with a familiarity vector of a trained vehicle data set, wherein the vector representation is a representation of effectiveness of the scene detection operation in identifying target object attributes of the vehicle sensor data,
identify significant scenario data based on the vector representation, wherein the significant scenario identifies at least one target object of the vehicle sensor data,
add the significant scenario data to a resultant data set, and
update a clustering of the trained vehicle data set based on the resultant data set.

13. The vehicle control unit of claim 12 , wherein the vehicle sensor data includes at least one of image, radar, and LiDAR data for a detection zone of a driver assistance system of the vehicle.

14. The vehicle control unit of claim 12 , wherein running the scene detection operation on the vehicle sensor data generates an annotated data set for target objects in real time based on the attributes of a trained vehicle data set, the trained vehicle data set providing a plurality of object types and object attributes.

15. The vehicle control unit of claim 12 , wherein comparing the vector representation includes performing a clustering operation for target objects of the vehicle sensor data using the trained vehicle data set to generate a vector data model for the vehicle sensor data, the vector data model characterizing ability of the trained vehicle set to perceive target objects of the vehicle sensor data.

16. The vehicle control unit of claim 12 , wherein identifying the significant scenario includes determining that a target object is an unidentified object.

17. The vehicle control unit of claim 12 , wherein identifying the significant scenario includes determining that the trained vehicle data set is unable to classify a target object.

18. The vehicle control unit of claim 12 , wherein identifying the significant scenario includes determining familiarity of a target object relative to the trained data set based on at least one of a number of target objects, classification of target objects, size and shape of target object, object type, and object color.

19. The vehicle control unit of claim 12 , wherein identifying the significant scenario includes determining at least one vehicle operation characteristic as an attribute relative to identification of a target object in at least one of a driver assistance system and autonomous driving system.

20. The vehicle control unit of claim 12 , wherein the vehicle control unit is further configured to score the vector representation on an ability of the scene detection operation to perceive target object attributes of the vehicle sensor data using the trained vehicle data set, and wherein the significant scenario is identified data based on a score of the vector representation below a predetermined threshold.

Assignees

• Harman Int Ind

Inventors

• Thompson Aaron
• Tan Honghao

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N3/08 Primary

  Learning methods · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• B60W2050/0005

  Processor details or data handling, e.g. memory registers or chip architecture · CPC title

• G06N20/00

  Machine learning · CPC title