System and method for determining vehicle data set familiarity

Patent metadata

Field	Value
Publication number	US-11688174-B2
Application number	US-202117361105-A
Country	US
Kind code	B2
Filing date	Jun 28, 2021
Priority date	Oct 10, 2018
Publication date	Jun 27, 2023
Grant date	Jun 27, 2023

Abstract

The present disclosure relates to systems, devices and methods for identifying objects and scenarios that have not been trained or are unidentifiable to vehicle perception sensors or vehicle assistive driving systems. Embodiments are directed to using a trained vehicle data set to identify target objects in vehicle sensor data. In one embodiment, a process is provided that includes running a scene detection operation on vehicle to derive a vector of target object attributes of the vehicle sensor data and generating a vector representation for the scene detection operation and the attributes of the vehicle sensor data. The vector representation compared to a familiarity vector to represent effectiveness of the scene detection operation. In addition, the vector representation can be scored to identify one or more target objects or significant scenarios, including unidentifiable objects and/or driving scenes, scenarios for reporting.

First claim

What is claimed is:

1. A vehicle control unit comprising:
an input configured to receive vehicle sensor data; and
a control unit coupled to the input, wherein the control unit is configured to:
receive vehicle sensor data captured by at least one sensor of the vehicle, the vehicle sensor data generated by at least one perception sensor of a vehicle;
run a scene detection operation on the vehicle sensor data to derive a vector of target object attributes of the vehicle sensor data;
compare the vector of target object attributes against a cluster of a vector data model of a trained vehicle data set; and
identify significant scenario data based on a divergence between the vector of target object attributes and the cluster, wherein the significant scenario data identifies at least one target object of the vehicle sensor data.

2. The vehicle control unit of claim 1 , wherein the vehicle sensor data includes at least one of image, radar, and LiDAR data for a detection zone of a driver assistance system of the vehicle.

3. The vehicle control unit of claim 1 , wherein running the scene detection operation on the vehicle sensor data generates an annotated data set for target objects in real time based on the attributes of the trained vehicle data set, the trained vehicle data set providing a plurality of object types and object attributes.

4. The vehicle control unit of claim 1 , wherein the cluster of the vector data model of the trained vehicle data set is generated by a k-means clustering algorithm.

5. The vehicle control unit of claim 1 , wherein identifying the significant scenario includes determining that a target object is not represented in the trained vehicle data set.

6. The vehicle control unit of claim 1 , wherein identifying the significant scenario includes determining that the trained vehicle data set is unable to classify a target object.

7. The vehicle control unit of claim 1 , wherein identifying the significant scenario includes determining familiarity of a target object relative to the trained data set based on at least one of a number of target objects, classification of target objects, size and shape of target object, object type, and object color.

8. The vehicle control unit of claim 1 , wherein identifying the significant scenario includes determining at least one vehicle operation characteristic as an attribute relative to identification of a target object in at least one of a driver assistance system and autonomous driving system.

9. The vehicle control unit of claim 1 , wherein identifying the significant scenario includes determining familiarity for a current scene of vehicle operation relative to a vehicle trained data set for at least one driving condition.

10. The vehicle control unit of claim 1 , wherein the control unit is further configured to score the vector on an ability of the scene detection operation to perceive target object attributes of the vehicle sensor data using the trained vehicle data set, and wherein the significant scenario is identified, by the control unit, based on a score of the vector of target object attributes being below a predetermined threshold.

11. The vehicle control unit of claim 1 , wherein the control unit is further configured to update a training of a run time algorithm in a subsequent release by updating a baseline data set.

12. A method for identifying significant scenario data by a control unit of a vehicle, the method comprising:
receiving, by a control unit, vehicle sensor data captured by at least one sensor of the vehicle, the vehicle sensor data generated by at least one perception sensor of a vehicle;
running, by the control unit, a scene detection operation on the vehicle sensor data to derive a vector of target object attributes of the vehicle sensor data;
comparing, by the control unit, a vector representation for the scene detection operation with a familiarity vector of a trained vehicle data set, wherein the vector representation is a representation of effectiveness of the scene detection operation in identifying target object attributes of the vehicle sensor data;
identifying, by the control unit, significant scenario data based on the vector representation, wherein the significant scenario identifies at least one target object of the vehicle sensor data;
adding, by the control unit, the significant scenario data to a resultant data set; and
updating, by the control unit, a clustering of the trained vehicle data set based on the resultant data set.

13. The method of claim 12 , wherein the vehicle sensor data includes at least one of image, radar, and LiDAR data for a detection zone of a driver assistance system of the vehicle.

14. The method of claim 12 , wherein running the scene detection operation on the vehicle sensor data generates an annotated data set for target objects in real time based on the attributes of a trained vehicle data set, the trained vehicle data set providing a plurality of object types and object attributes.

15. The method of claim 12 , wherein comparing the vector representation includes performing a clustering operation for target objects of the vehicle sensor data using the trained vehicle data set to generate a vector data model for the vehicle sensor data, the vector data model characterizing ability of the trained vehicle set to perceive target objects of the vehicle sensor data.

16. The method of claim 12 , wherein identifying the significant scenario includes determining that a target object is an unidentified object.

17. The method of claim 12 , wherein identifying the significant scenario includes determining that the trained vehicle data set is unable to classify a target object.

18. The method of claim 12 , wherein identifying the significant scenario includes determining familiarity of a target object relative to the trained data set based on at least one of a number of target objects, classification of target objects, size and shape of target object, object type, and object color.

19. The method of claim 12 , wherein identifying the significant scenario includes determining at least one vehicle operation characteristic as an attribute relative to identification of a target object in at least one of a driver assistance system and autonomous driving system.

20. The method of claim 12 , further comprising scoring, by the control unit, the vector representation on an ability of the scene detection operation to perceive target object attributes of the vehicle sensor data using the trained vehicle data set, and wherein the significant scenario is identified by the control unit based on a score of the vector representation being below a predetermined threshold.

Assignees

• Harman Int Ind

Inventors

• Thompson Aaron
• Tan Honghao

Classifications

• G06N3/09

  Supervised learning · CPC title

• G06N20/00

  Machine learning · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• B60W50/00 Primary

  Details of control systems for road vehicle drive control not related to the control of a particular sub-unit {, e.g. process diagnostic or vehicle driver interfaces} · CPC title