Virtual sensor data generation for wheel stop detection

Patent metadata

Field	Value
Publication number	US-9740944-B2
Application number	US-201514975177-A
Country	US
Kind code	B2
Filing date	Dec 18, 2015
Priority date	Dec 18, 2015
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

The disclosure relates to methods, systems, and apparatuses for virtual sensor data generation and more particularly relates to generation of virtual sensor data for training and testing models or algorithms to detect objects or obstacles, such as wheel stops or parking barriers. A method for generating virtual sensor data includes simulating a three-dimensional (3D) environment comprising one or more objects. The method includes generating virtual sensor data for a plurality of positions of one or more sensors within the 3D environment. The method includes determining virtual ground truth corresponding to each of the plurality of positions, wherein the ground truth includes information about at least one object within the virtual sensor data. The method also includes storing and associating the virtual sensor data and the virtual ground truth.

First claim

What is claimed is:

1. A method comprising:
simulating, using one or more processors, a three-dimensional (3D) environment comprising one or more parking barriers;
generating, using the one or more processors, virtual sensor data for a plurality of positions of one or more sensors within the 3D environment;
determining, using the one or more processors, virtual ground truth corresponding to each of the plurality of positions, wherein the ground truth comprises a height of the at least one of the parking barriers; and
storing and associating the virtual sensor data and the virtual ground truth using the one or more processors.

2. The method of claim 1 , further comprising providing one or more of the virtual sensor data and the virtual ground truth for training or testing of a machine learning algorithm or model.

3. The method of claim 2 , wherein the machine learning model or algorithm comprises a neural network.

4. The method of claim 2 , wherein training the machine learning algorithm or model comprises providing at least a portion of the virtual sensor data and corresponding virtual ground truth to train the machine learning algorithm or model to determine one or more of a height and a position of the one or more parking barriers.

5. The method of claim 2 , wherein testing the machine learning algorithm or model comprises providing at least a portion of the virtual sensor data to the machine learning algorithm or model to determine one or more of a height and a position of the parking barrier and compare a determined height or a determined position with the virtual ground truth.

6. The method of claim 1 , wherein the plurality of positions correspond to planned locations of sensors on a vehicle.

7. The method of claim 1 , wherein the virtual sensor data comprises one or more of computer generated images, computer generated radar data, computer generated LIDAR data, and computer generated ultrasound data.

8. The method of claim 1 , wherein simulating the 3D environment comprises randomly generating different conditions for one or more of lighting, weather, a position of the one or more virtual parking barriers, and dimensions of the one or more virtual parking barriers.

9. The method of claim 1 , wherein generating the virtual sensor data comprises periodically generating the virtual sensor data during simulated movement of the one or more sensors within the 3D environment.

10. The method of claim 1 , wherein determining the virtual ground truth comprises generating a ground truth frame complimentary to a frame of virtual sensor data, wherein the ground truth frame comprises a same color value for pixels corresponding to the one or more virtual parking barriers.

11. A system comprising:
an environment component configured to simulate, using one or more processors, a three-dimensional (3D) environment comprising one or more virtual parking barriers;
a virtual sensor component configured to generate, using the one or more processors, virtual sensor data for a plurality of positions of one or more sensors within the 3D environment;
a ground truth component configured to determine virtual ground truth corresponding to each of the plurality of positions, wherein the ground truth comprises a height of the at least one of the virtual parking barriers; and
a model component configured to provide the virtual perception data and the ground truth to a machine learning model or algorithm to train or test the machine learning model or algorithm using the one or more processors.

12. The system of claim 11 , wherein the model component is configured to train the machine learning algorithm or model, wherein training comprises:
providing at least a portion of the virtual sensor data and corresponding virtual ground truth to train the machine learning algorithm or model to determine one or more of a height and a position of the one or more parking barriers.

13. The system of claim 11 , wherein the model component is configured to test the machine learning algorithm or model, wherein testing comprises:
providing at least a portion of the virtual sensor data to the machine learning algorithm or model to determine one or more of a height and a position of the parking barrier; and
comparing a determined height or a determined position with the virtual ground truth.

14. The system of claim 11 , wherein the virtual sensor component is configured to generate virtual sensor data comprising one or more of computer generated images, computer generated radar data, computer generated light detection and ranging (LIDAR) data, and computer generated ultrasound data.

15. The system of claim 11 , wherein the environment component is configured to simulate the 3D environment by randomly generating different conditions for one or more of the plurality of positions, wherein the different conditions comprise one or more of:
lighting conditions;
weather conditions;
a position of the one or more virtual parking barriers; and
dimensions of the one or more virtual parking barriers.

16. Non-transitory computer readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to:
generate virtual sensor data for a plurality of sensor positions within a simulated three-dimensional (3D) environment comprising one or more virtual parking barriers;
determine one or more simulated conditions for each of the plurality of positions, wherein the simulated conditions comprise one or more of a height of the at least one of the virtual parking barriers and a position of the virtual parking barriers in relation to a virtual sensor; and
store and annotate the virtual sensor data with the simulated conditions.

17. The computer readable storage of claim 16 , wherein the instructions further cause the one or more processors to train or test a machine learning algorithm or model based on one or more of the virtual sensor data and the simulated conditions.

18. The computer readable storage of claim 17 , wherein one or more of:
the instructions cause the one or more processors to train the machine learning algorithm or model by providing at least a portion of the virtual sensor data and corresponding simulated conditions to train the machine learning algorithm or model to determine one or more of a height and a position of the one or more parking barriers; and
the instructions cause the one or more processors to test the machine learning algorithm or model by providing at least a portion of the virtual sensor data to the machine learning algorithm or model to determine one or more of a height and a position of the parking barrier and by comparing a determined height or a determined position with the simulated conditions.

19. The computer readable storage of claim 16 , wherein generating the virtual sensor data comprises simulating the 3D environment by randomizing one or more of the simulated conditions for one or more of the plurality of positions, wherein randomizing the one or more simulated conditions comprises randomizing one or more of:
lighting conditions;
weather conditions;
a position of the one or more virtual parking barriers; and
dimensions of the one or more virtual parking barriers.

20. The computer readable storage of claim 16 , wherein determining the simulated conditions further comprises generating a ground truth frame complimentary to a frame of virtual sensor data, wherein the ground truth frame comprises a same color value for pixels corresponding to the one or more virtua

Assignees

• Ford Global Tech Llc

Inventors

• Micks Ashley Elizabeth
• Nallapa Venkatapathi Raju
• Banvait Harpreetsingh
• Myers Scott Vincent

Classifications

• G06V10/774 Primary

  Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06V20/58

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• B60W2554/00

  Input parameters relating to objects · CPC title

• G06T7/251

  involving models · CPC title