Virtual sensor data generation for wheel stop detection

Patent metadata

Field	Value
Publication number	US-10635912-B2
Application number	US-201715627108-A
Country	US
Kind code	B2
Filing date	Jun 19, 2017
Priority date	Dec 18, 2015
Publication date	Apr 28, 2020
Grant date	Apr 28, 2020

Abstract

The disclosure relates to methods, systems, and apparatuses for virtual sensor data generation and more particularly relates to generation of virtual sensor data for training and testing models or algorithms to detect objects or obstacles. A method for generating virtual sensor data includes simulating, using one or more processors, a three-dimensional (3D) environment comprising one or more virtual objects. The method includes generating, using one or more processors, virtual sensor data for a plurality of positions of one or more sensors within the 3D environment. The method includes determining, using one or more processors, virtual ground truth corresponding to each of the plurality of positions, wherein the ground truth comprises a dimension or parameter of the one or more virtual objects. The method includes storing and associating the virtual sensor data and the virtual ground truth using one or more processors.

First claim

What is claimed is:

1. A computer implemented method comprising:
simulating a three-dimensional (3D) environment comprising a plurality of static virtual objects and a driving surface;
randomizing dimensions and positions of the plurality of static virtual objects within the 3D environment;
simulating a sensor traveling the 3D environment, wherein the sensor is positioned relative to the driving surface according to a planned position of the sensor on a vehicle;
recording virtual sensor data captured by the sensor as the sensor travels the 3D environment; and
annotating the virtual sensor data with ground truth about the dimensions and the positions of the static virtual objects that are present in the virtual sensor data.

2. The method of claim 1 , further comprising generating a dataset comprising annotated virtual sensor data for training or testing a machine learning algorithm or model.

3. The method of claim 2 , further comprising generating a plurality of datasets having a plurality of randomized environmental conditions across a plurality of 3D environments.

4. The method of claim 2 , further comprising training the machine learning algorithm or model by providing at least a portion of the virtual sensor data and corresponding ground truth to train the machine learning algorithm or model to determine one or more of a dimension or position of a static virtual object.

5. The method of claim 2 , further comprising testing the machine learning algorithm or model by providing at least a portion of the virtual sensor data to the machine learning algorithm or model to determine one or more of a dimension or position of a static virtual object and compare a determined dimension or a determined position with the ground truth corresponding to the virtual sensor data.

6. The method of claim 1 , wherein the plurality of static virtual objects are parking barriers and the driving surface is a parking lot surface.

7. The method of claim 1 , wherein the virtual sensor data comprises one or more of computer generated images, computer generated radar data, computer generated LIDAR data, and computer generated ultrasound data.

8. The method of claim 1 , further comprising randomizing a lighting condition for the plurality of static virtual objects within the 3D environment.

9. The method of claim 1 , wherein the virtual sensor data comprises computer generated images and the method further comprises generating a complimentary image frame of pixelwise segmented image data such that all pixels belonging to a static virtual object are a solid color with a constant color value.

10. The method of claim 1 , further comprising generating complimentary sensor data corresponding to the virtual sensor data, wherein a static virtual object captured in the complimentary sensor data is depicted with a constant color value.

11. A system comprising:
a processor that is programmable to execute instructions stored in non-transitory computer readable storage media, the instructions comprising:
simulating a three-dimensional (3D) environment comprising a plurality of virtual parking barriers and a driving surface;
randomizing dimensions and positions of the plurality of virtual parking barriers within the 3D environment;
simulating a sensor traveling the 3D environment, wherein the sensor is positioned relative to the driving surface according to a planned position of the sensor on a vehicle;
recording virtual sensor data captured by the sensor as the sensor travels the 3D environment; and
annotating the virtual sensor data with ground truth about the dimensions and the positions of the virtual parking barriers that are present in the virtual sensor data.

12. The system of claim 11 , wherein the instructions further comprise generating a dataset comprising annotated virtual sensor data for training or testing a machine learning algorithm or model.

13. The system of claim 12 , wherein the instructions further comprise testing the machine learning algorithm or model by:
providing at least a portion of the virtual sensor data to the machine learning algorithm or model to determine one or more of a dimension or a position of a virtual parking barrier; and
comparing a determined dimension or a determined position with the ground truth.

14. The system of claim 11 , wherein the virtual sensor data comprises one or more of computer generated images, computer generated radar data, computer generated light detection and ranging (LIDAR) data, and computer generated ultrasound data.

15. The system of claim 11 , wherein the instructions further comprise randomizing a lighting condition for the plurality of virtual parking barriers within the 3D environment.

16. Non-transitory computer readable storage media storing instructions to be executed by one or more processors, the instructions comprising:
simulating a three-dimensional (3D) environment comprising a plurality of virtual parking barriers and a driving surface;
randomizing dimensions and positions of the plurality of virtual parking barriers within the 3D environment;
simulating a sensor traveling the 3D environment, wherein the sensor is positioned relative to the driving surface according to a planned position of the sensor on a vehicle;
recording virtual sensor data captured by the sensor as the sensor travels the 3D environment; and
annotating the virtual sensor data with ground truth about the dimensions and the positions of the virtual parking barriers that are present in the virtual sensor data.

17. The non-transitory computer readable storage of claim 16 , wherein the instructions further comprise generating a dataset comprising annotated virtual sensor data for training or testing a machine learning algorithm or model.

18. The non-transitory computer readable storage of claim 17 , wherein one or more of:
the instructions further comprising training the machine learning algorithm or model by providing at least a portion of the virtual sensor data and corresponding ground truth to train the machine learning algorithm or model to determine one or more of a dimension or a position of the one or more virtual parking barriers; and
the instructions further comprise testing the machine learning algorithm or model by providing at least a portion of the virtual sensor data to the machine learning algorithm or model to determine one or more of a dimension or a position of the one or more virtual parking barriers and by comparing a determined dimension or a determined position with the ground truth.

19. The non-transitory computer readable storage of claim 16 , wherein the instructions further comprise randomizing a lighting condition for the plurality of virtual parking barriers within the 3D environment.

20. The non-transitory computer readable storage of claim 16 , wherein the virtual sensor data comprises computer generated images and the instructions further comprise generating a complimentary image frame of pixelwise segmented image data such that all pixels belonging to a virtual parking barrier captured in the complimentary image frame are a solid color with a constant color value.

Assignees

• Ford Global Tech Llc

Inventors

• Micks Ashley Elizabeth
• Nallapa Venkatapathi Raju
• Banvait Harpreetsingh
• Myers Scott Vincent

Classifications

• G08G1/168

  Driving aids for parking, e.g. acoustic or visual feedback on parking space · CPC title

• G06T17/05

  Geographic models · CPC title

• B60W2554/00

  Input parameters relating to objects · CPC title

• B60W2400/00

  Indexing codes relating to detected, measured or calculated conditions or factors · CPC title

• B60W30/08

  Active safety systems} predicting or avoiding probable or impending collision {or attempting to minimise its consequences · CPC title