Conversion of measured data between measurement modalities

Patent metadata

Field	Value
Publication number	US-12092741-B2
Application number	US-202117355863-A
Country	US
Kind code	B2
Filing date	Jun 23, 2021
Priority date	Jun 25, 2020
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

A method for converting measured data of at least one source measurement modality into realistic measured data of at least one target measurement modality. The method includes: the measured data of the source measurement modality are mapped onto representations in a latent space using an encoder of a trained encoder-decoder arrangement, and the representations are mapped onto the realistic measured data of the target measurement modality using the decoder of the encoder-decoder arrangement, the amount of information of the representations of measured data in the latent space being smaller than the amount of information of the measured data.

First claim

What is claimed is:

1. A method for converting measured data of at least one source measurement modality into realistic measured data of at least one target measurement modality, comprising the following steps:
mapping the measured data of the source measurement modality onto representations in a latent space using an encoder of a trained encoder-decoder arrangement; and
mapping the representations onto the realistic measured data of the target measurement modality using the decoder of the encoder-decoder arrangement, wherein an amount of information of the representations of measured data in the latent space is smaller than an amount of information of the measured data, wherein a dimensionality of the latent space is both smaller than a dimensionality of a space from which the encoder obtains the measured data of the source measurement modality, as well as smaller than a dimensionality of a space into which the decoder maps the measured data of the target measurement modality.

2. The method as recited in claim 1 , wherein the measured data is measured data of multiple measurement modalities and are mapped by respective encoders onto representations in the latent space, and the representations are mapped by the same decoder onto measured data of the target measurement modality.

3. The method as recited in claim 1 , wherein at least one representation in the latent space is mapped by at least two different decoders onto measured data of at least two target measurement modalities.

4. A method for converting measured data of at least one source measurement modality into realistic measured data of at least one target measurement modality, comprising the following steps:
mapping the measured data of the source measurement modality onto representations in a latent space using an encoder of a trained encoder-decoder arrangement; and
mapping the representations onto the realistic measured data of the target measurement modality using the decoder of the encoder-decoder arrangement, wherein an amount of information of the representations of measured data in the latent space is smaller than an amount of information of the measured data, wherein the measured data of the source measurement modality contain a description of a setting including one or multiple objects, and the measured data of the target measurement modality includes locations in space, which a specific sensor, when physically observing the setting, would assign in each case to the objects in the setting.

5. The method as recited in claim 4 , wherein the measured data of the target measurement modality includes locations in space at which a specific radar sensor or a specific LIDAR sensor, when physically observing the setting, would register a radar reflex or a LIDAR reflex.

6. The method as recited in claim 4 , wherein the description of the setting encompasses definitions of a plurality of geometric shapes in a plane or in space, each of which is considered to be occupied by an object.

7. The method as recited in claim 6 , wherein:
the measured data of the source measurement modality are transformed into an input image or into an input point cloud;
the input image or the input point cloud is mapped by the encoder onto a representation in the latent space;
the representation is mapped by the decoder onto an output image or onto an output point cloud; and
the output image and/or the output point cloud are transformed into the measured data of the target measurement modality.

8. The method as recited in claim 7 , wherein the description of the setting is transformed into an input image by discretizing the setting using a two-dimensional or three-dimensional grid and checking each grid point for whether it belongs to one of the geometric shapes defined in the description.

9. The method as recited in claim 8 , wherein the grid points of the input image or the points of the input point cloud are assigned additional pieces of information relating to the material, to a class, and/or to a movement state of the object from the description of the setting.

10. The method as recited in claim 7 , wherein the description of the setting is transformed into an input point cloud by drawing coordinates of points from a distribution and by adding each point that belongs to one of the geometric shapes to the input point cloud.

11. A method for checking a control logic for a driving assistance system or a system for at least the semi-automated driving of a vehicle, the method comprising the following steps:
providing a description of a setting including a predefined arrangement of objects as measured data of a source measurement modality;
transforming the measured data of the source measurement modality into measured data of a target measurement modality;
feeding the measured data of the target measurement modality as input to the control logic to be checked;
comparing a response proposed by the control logic based on the input with a setpoint response predefined for the setting;
based on a result of the comparison, evaluating to what extent the control logic is able to master the setting.

12. A method for training an encoder-decoder arrangement, comprising the following steps:
providing learning source measured data of at least one source measurement modality, the learning source measured data representing predefined physical settings;
providing learning target measured data of at least one measurement modality for the same physical settings at the predefine physical settings;
initially mapping the learning source measured data by the encoder-decoder arrangement onto representations in a latent space and from there onto measured data of the target measurement modality;
assessing, according to a measure of one predefined cost function, how well the measured data of the target measurement modality match the learning target measured data;
optimizing parameters that characterize a behavior of the encoder, and/or parameters that characterize a behavior of the decoder, the optimizing being with an aim that with further processing of learning source measured data by the encoder-decoder arrangement, the assessment of the then resultant measured data of the target measurement modality is improved.

13. The method as recited in claim 12 , wherein the learning target measured data are transformed into a learning target image or into a learning target point cloud, and a match between an output image generated by the decoder or an output point cloud generated by the decoder and the learning target image or the learning target point cloud is graded as the match between the measured data of the target measurement modality and the learning target measured data.

14. A non-transitory machine-readable data medium on which is stored a computer program for converting measured data of at least one source measurement modality into realistic measured data of at least one target measurement modality, the computer program, when executed by a computer, causing the computer to perform the following steps:
mapping the measured data of the source measurement modality onto representations in a latent space using an encoder of a trained encoder-decoder arrangement; and
mapping the representations onto the realistic measured data of the target measurement modality using the decoder of the encoder-decoder arrangement, wherein an amount of information of the representations of measured data in the latent space is smaller than an amount of information of the measured data, wherein a dimensionality of the latent space is both smaller than a dimensionality of a space from which the encoder obtains the measured data of the source me

Assignees

• Bosch Gmbh Robert

Inventors

• Junginger Andrej

Classifications

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06F11/3698

  Environments for analysis, debugging or testing of software · CPC title