Physics modeling for radar and ultrasonic sensors

Patent metadata

Field	Value
Publication number	US-10592805-B2
Application number	US-201615248787-A
Country	US
Kind code	B2
Filing date	Aug 26, 2016
Priority date	Aug 26, 2016
Publication date	Mar 17, 2020
Grant date	Mar 17, 2020

Abstract

A machine learning module may generate a probability distribution from training data including labeled modeling data correlated with reflection data. Modeling data may include data from a LIDAR system, camera, and/or a GPS for a target environment/object. Reflection data may be collected from the same environment/object by a radar and/or an ultrasonic system. The probability distribution may assign reflection coefficients for radar and/or ultrasonic systems conditioned on values for modeling data. A mapping module may create a reflection model to overlay a virtual environment assembled from a second set of modeling data by applying the second set to the probability distribution to assign reflection values to surfaces within the virtual environment. Additionally, a test bench may evaluate an algorithm, for processing reflection data to generate control signals to an autonomous vehicle, with simulated reflection data from a virtual sensor engaging reflection values assigned within the virtual environment.

First claim

The invention claimed is:

1. A system, comprising:
a processor set configured to process a set of training data, the set of training data comprising:
three-dimensional modeling data from at least one of a camera system, a LIght-Detection-And-Ranging (LIDAR) system, and a locational-awareness system; and
physics modeling data from at least one of a radar system and an ultrasonic system;

a machine learning module which, when executed by the processor set, learns a mapping to physical modeling characteristics from three-dimensional modeling information;
a mapping module which, when executed by the processor set, creates a physical model for a virtual environment by applying the mapping to the three-dimensional modeling information, the physical model providing reflection values for at least one of a virtual radar and a virtual ultrasonic system in the virtual environment; and
a model generation module which, when executed by the processor set, assembles the three-dimensional modeling information into the three-dimensional model of the virtual environment,
wherein the mapping module, when executed by the processor set, also applies the mapping to the three-dimensional modeling information by assigning reflectivity coefficients to surfaces in the three-dimensional model of the virtual environment for reflectivity of each of the surfaces.

2. The system of claim 1 , wherein
the three-dimensional modeling information carries information with which to generate a three-dimensional model of a virtual environment.

3. The system of claim 2 , further comprising a test bench, implemented on the processor set, operable to implement a test algorithm over a path through the virtual environment, the test algorithm processing reflection values received from the virtual environment throughout the path and providing outcomes in the form of at least one of detections, classifications, and determinations.

4. The system of claim 2 , further comprising a set of test benches, implemented on the processor set, comprising:
at least one Hardware-In-Loop (HIL) system operable to provide mathematical models for mechanical dynamics of a virtual vehicle as it travels a path through the virtual environment and dynamics for the at least one of the virtual radar and the virtual ultrasonic system; and
a Software-In-Loop (SIL) system communicably coupled to the at least one HIL system and operable to implement a test algorithm over the path through the virtual environment, the test algorithm processing reflection values received from the virtual environment throughout the path and providing control signals to the virtual vehicle.

5. The system of claim 1 , further comprising:
a model generation module which, when executed by the processor set, assembles the three-dimensional modeling information into a three-dimensional model of a virtual object; and wherein
the mapping module, when executed by the processor set, is further operable to apply the mapping to the three-dimensional modeling information by assigning reflectivity coefficients to surfaces of the three-dimensional model of the virtual object for reflectivity of each of the surfaces of the virtual objects, creating the virtual object for positioning in a virtual environment with a reflection cross-section for at least one of the virtual radar and the virtual ultrasonic system relative to the virtual object.

6. The system of claim 1 , wherein:
the set of training data is indexed to labels for a set of target categories; and
the machine learning module comprises a deep neural network operable to be trained with supervised learning using the labels indexed to the set of training data.

7. The system of claim 6 , wherein the deep neural network further comprises a classifier implemented with a convolution neural network.

8. The system of claim 1 , further comprising a heuristic module within the machine learning module, and wherein, when executed by the processor set, identifies aspects of the three-dimensional modeling data subject to a set of heuristics informing the machine learning module.

9. A method for physical modeling, further comprising:
applying a machine learning algorithm to a set of training data to create a probability distribution for reflection coefficients conditioned on a set of modeling data of surface reflectivity acquired by at least one of a camera system, a LIght-Detection-And-Ranging (LIDAR) system, and a position system; and
assigning reflection values to the set of modeling data by applying the set of modeling data to the probability distribution using a mapping function,
wherein the assigning the reflection values comprises assigning the reflection values to a set of surface regions for reflectivity of each of the set of surface regions, and
wherein the applying the machine learning algorithm to the set of training data comprises:
performing supervised learning on a deep neural network with the training data; and
after training, the deep neural network implements the probability distribution.

10. The method of claim 9 , further comprising:
assembling a set of surface regions from the set of modeling data to model a simulation environment simulating an environment from which the set of modeling data is captured;
the step of assigning the reflection values further comprises:
collecting reflection data from transmissions from a virtual sensor placed within the simulation environment, as reflected in accordance with the reflection values assigned to the set of surface regions in the simulation environment; and
testing a perception algorithm on the reflection data.

11. The method of claim 10 , wherein testing the perception algorithm further comprises:
placing a virtual vehicle within the simulation environment, coupled to the virtual sensor and operable to travel a path through the simulation environment; and
sending control signals output by the perception algorithm to the virtual vehicle.

12. The method of claim 9 , further comprising:
assembling a set of surface regions from the set of modeling data to form a virtual object;
the step of assigning the reflection values further comprises:
placing the virtual object within a virtual environment;
collecting reflection data from transmissions from a virtual sensor placed within the virtual environment, as reflected in accordance with the reflection values assigned to the set of surface regions in the simulation environment; and
testing a perception algorithm on the reflection data.

13. The method of claim 9 , further comprising tagging portions of at least one of the set of training data and the set of modeling data, which portions correspond to regions in at least one environment from which the at least one of the set of training data and the set of modeling data is collected, and which regions are correlated with ranges of the reflection values.

14. The method of claim 9 , further comprising collecting the set of training data, which further comprises:
collecting a training set of modeling data for a set of target areas;
collecting a training set of reflection data from the set of target areas correlated with the training set of modeling data;
identifying aspects of the set of target areas in the training data with a set of labels for supervised learning with the set of training data.

15. A system for modeling reflections, comprising:
a processor set configured to process a set of virtual-environment data, which is captured by at least one of a camera system, a Light-Detection-And-Ranging (LIDAR) system, and a locational-awareness system, sufficient to generate a three-dimension

Assignees

• Ford Global Tech Llc

Inventors

• Groh Alexander
• Kunes Kay
• Houts Sarah

Classifications

• G06F2111/08

  Probabilistic or stochastic CAD · CPC title

• G06F30/15

  Vehicle, aircraft or watercraft design · CPC title

• G01S15/08

  Systems for measuring distance only (indirect measurement G01S15/46) · CPC title

• G01S13/865

  Combination of radar systems with lidar systems · CPC title

• G09B9/54 Primary

  Simulation of radar (G09B9/40 takes precedence) · CPC title