Virtual sensor-data-generation system and method supporting development of algorithms facilitating navigation of railway crossings in varying weather conditions

Patent metadata

Field	Value
Publication number	US-11210436-B2
Application number	US-201615204857-A
Country	US
Kind code	B2
Filing date	Jul 7, 2016
Priority date	Jul 7, 2016
Publication date	Dec 28, 2021
Grant date	Dec 28, 2021

Abstract

A method for generating training data is disclosed. The method may include executing a simulation process. The simulation process may include traversing a virtual, forward-looking sensor over a virtual road surface defining at least one virtual railroad crossing. During the traversing, the virtual sensor may be moved with respect to the virtual road surface as dictated by a vehicle-motion model modeling motion of a vehicle driving on the virtual road surface while carrying the virtual sensor. Virtual sensor data characterizing the virtual road surface may be recorded. The virtual sensor data may correspond to what a real sensor would have output had it sensed the road surface in the real world.

First claim

What is claimed is:

1. A method for generating data usable in developing, training, or proving algorithms for negotiating railroad crossings, the method comprising:
traversing, by a computer system, one or more virtual sensors over a virtual road surface in a simulation;
recording, by the computer system, data corresponding to signals output by the one or more virtual sensors during the traversing, wherein the data characterizes a virtual railroad crossing in the virtual road surface; and
annotating, by the computer system, the data with ground-truth data corresponding to the virtual railroad crossing, wherein the ground-truth data includes a relative orientation of the virtual railroad crossing.

2. The method of claim 1 , wherein the virtual railroad crossing is one of a plurality of virtual railroad crossings distributed cross the virtual road surface, each virtual railroad crossing of the plurality of virtual railroad crossings defining a structure sensible by the a first sensor of the one or more virtual sensors.

3. The method of claim 2 , wherein the traversing comprises moving each of the one or more virtual sensors with respect to the virtual road surface as dictated by a vehicle-motion model modeling motion of a vehicle carrying the one or more virtual sensors and driving on the virtual road surface.

4. The method of claim 3 , wherein the one or more virtual sensors comprise a forward-looking sensor positioned to sense a portion of the virtual road surface ahead of the vehicle.

5. The method of claim 4 , wherein the forward-looking sensor is a camera, radar, or lidar device.

6. The method of claim 5 , wherein the data further characterizes weather conditions proximate the virtual road surface.

7. The method of claim 6 , wherein the annotating further comprises annotating the data with ground-truth information corresponding to the weather conditions proximate the virtual road surface.

8. The method of claim 7 , wherein a second virtual sensor comprises a thermometer.

9. The method of claim 8 , wherein the data characterizes an ambient temperature proximate the virtual road surface.

10. The method of claim 9 , wherein the annotating further comprises annotating the data with ground-truth information comprising at least one temperature value proximate the virtual road surface.

11. A method of generating training data, the method comprising:
executing, by a computer system, a simulation comprising
traversing one or more virtual sensors over a virtual road surface, the virtual road surface defining one or more virtual railroad crossings that are each sensible by a first sensor of the one or more virtual sensors, and
moving, during the traversing, each of the one or more virtual sensors with respect to the virtual road surface as dictated by a vehicle-motion model modeling motion of a vehicle driving on the virtual road surface while carrying the one or more virtual sensors;

recording, by the computer system, perception data characterizing the one or more virtual railroad crossings, the data corresponding to signal output by the first virtual sensor during the traversing; and
annotating, by the computer system, the perception data with ground-truth data corresponding to each of the one or more virtual railroad crossings, wherein the ground-truth data includes a relative orientation of each virtual railroad crossing of the one or more virtual railroad crossing.

12. The method of claim 11 , wherein the relative orientation numerically characterizes an angle at which a corresponding virtual railroad crossing of the one or more virtual railroad crossings crosses the virtual road surface.

13. The method of claim 11 , wherein the first virtual sensor comprises a forward-looking sensor positioned to sense a portion of the virtual road surface ahead of the vehicle.

14. The method of claim 13 , wherein the forward-looking sensor is a camera, radar, or lidar device.

15. The method of claim 14 , wherein the perception data further characterizes weather conditions proximate the virtual road surface.

16. The method of claim 15 , wherein the annotating further comprises annotating the perception data with ground-truth data corresponding to the weather conditions proximate the virtual road surface.

17. The method of claim 16 , wherein a second virtual sensor comprises a thermometer.

18. The method of claim 17 , wherein the perception data characterizes an ambient temperature proximate the virtual road surface.

19. The method of claim 18 , wherein the annotating further comprises annotating the perception data with ground-truth data comprising at least one temperature value proximate the virtual road surface.

20. A computer system comprising:
one or more processors;
memory operably connected to the one or more processors; and
the memory storing
a virtual driving environment programmed to include a plurality of virtual railroad crossings,
a first software model programmed to model a forward-looking sensor,
a second software model programmed to model a vehicle,
a simulation module programmed to use the virtual driving environment, the first software model, and the second software model to produce an output modeling what would be output by the forward-looking sensor had the forward-looking sensor been mounted to the vehicle and the vehicle had driven on an actual driving environment matching the virtual driving environment, and
the simulation module further programmed to annotate the output with ground-truth data corresponding to the plurality of virtual railroad crossings, wherein the ground-truth data includes a relative orientation of each virtual railroad crossing of the plurality of virtual railroad crossings.

Assignees

• Ford Global Tech Llc

Inventors

• Micks Ashley Elizabeth
• Myers Scott Vincent
• Banvait Harpreetsingh
• Kadetotad Sneha

Classifications

• G06F18/21 Primary

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06F30/20 Primary

  Design optimisation, verification or simulation (optimisation, verification or simulation of circuit designs G06F30/30) · CPC title

• G06F30/15 Primary

  Vehicle, aircraft or watercraft design · CPC title

• G06N20/00

  Machine learning · CPC title

• G06V20/588

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title