Feature compression and localization for autonomous devices

Patent metadata

Field	Value
Publication number	US-11715012-B2
Application number	US-201916598561-A
Country	US
Kind code	B2
Filing date	Oct 10, 2019
Priority date	Nov 16, 2018
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

Systems, methods, tangible non-transitory computer-readable media, and devices associated with object localization and generation of compressed feature representations are provided. For example, a computing system can access source data and target data. The source data can include a source representation of an environment including a source object. The target data can include a compressed target feature representation of the environment. The compressed target feature representation can be based on compression of a target feature representation of the environment produced by machine-learned models. A source feature representation can be generated based on the source representation and the machine-learned models. The machine-learned models can include machine-learned feature extraction models or machine-learned attention models. A localized state of the source object with respect to the environment can be determined based on the source feature representation and the compressed target feature representation.

First claim

What is claimed is:

1. A computer-implemented method for localization of objects, the computer-implemented method comprising:
accessing source data and target data, the source data comprising a source representation of an environment comprising a source object, the target data comprising a compressed target feature representation of the environment, wherein the compressed target feature representation is based at least in part on compression of a target feature representation of the environment produced by one or more machine-learned feature extraction models;
generating a source feature representation based at least in part on the source representation and the one or more machine-learned feature extraction models; and
generating a decompressed target feature representation based at least in part on one or more lossless binary decoding operations;
generating a reconstructed target feature representation based at least in part on the decompressed target feature representation and a machine-learned decoding model; and
determining a localized state of the source object with respect to the environment based at least in part on one or more comparisons of the source feature representation to the reconstructed target feature representation.

2. The computer-implemented method of claim 1 , wherein the determining the localized state of the source object with respect to the environment based at least in part on the source feature representation and the compressed target feature representation comprises:
generating a reconstructed target feature representation based at least in part on the compressed target feature representation and a machine-learned reconstruction model, wherein the reconstructed target feature representation is a reconstruction of the target feature representation; and
determining the localized state of the source object based at least in part on one or more comparisons of the source feature representation to the reconstructed target feature representation.

3. The computer-implemented method of claim 2 , wherein the determining the localized state of the source object based at least in part on one or more comparisons of the source feature representation to the reconstructed target feature representation comprises:
determining one or more correlations between the reconstructed target feature representation and the source feature representation based at least in part on a probabilistic inference model configured to encode agreement between the source feature representation and the reconstructed target feature representation indexed at the localized state of the source object.

4. The computer-implemented method of claim 2 , wherein the compressed target feature representation is based at least in part on an encoding of the target feature representation using one or more lossless compression operations, and wherein the generating the reconstructed target feature representation based at least in part on the compressed target feature representation and the machine-learned reconstruction model, wherein the reconstructed target feature representation is a reconstruction of the target feature representation comprises:
generating a decoded target feature representation of the compressed target feature representation based at least in part on the one or more lossless compression operations, wherein the one or more lossless compression operations comprise one or more lossless binary encoding operations; and
generating the target feature representation based at least in part on the decoded target feature representation and the machine-learned reconstruction model.

5. The computer-implemented method of claim 1 , wherein the determining the localized state of the source object with respect to the environment based at least in part on the source feature representation and the compressed target feature representation comprises:
rotating the source feature representation to a plurality of candidate angles; and
determining at the plurality of candidate angles, whether the source feature representation matches the compressed target feature representation.

6. The computer-implemented method of claim 1 , wherein the compressed target feature representation of the environment is based at least in part on an attended feature representation of the target feature representation generated by a machine-learned attention model configured to mask one or more portions of the target feature representation.

7. The computer-implemented method of claim 1 , wherein the source data is based at least in part on one or more sensor outputs from one or more sensors comprising at least one of: one or more light detection and ranging (LiDAR) devices, one or more sonar devices, one or more radar devices, or one or more cameras.

8. The computer-implemented method of claim 1 , wherein the one or more machine-learned feature extraction models comprise a first machine-learned extraction model configured to generate the source feature representation and a second machine-learned model configured to generate the target feature representation.

9. A computing system comprising:
one or more processors;
one or more machine-learned feature extraction models configured to access training data comprising one or more representations of a training environment and generate one or more feature extracted representations of the training environment; and
one or more tangible non-transitory computer-readable media storing computer-readable instructions that are executable by one or more processors to cause the one or more processors to perform operations, the operations comprising:
accessing training data comprising a source representation of the training environment and a target representation of the training environment, wherein the source representation is associated with a ground-truth state of a source object in the training environment;
generating a source feature representation and a target feature representation based at least in part on the one or more machine-learned feature extraction models accessing the source representation and the target representation respectively;
generating a compressed target feature representation of the target feature representation based at least in part on one or more machine-learned compression models;
generating a decompressed target feature representation based at least in part on one or more lossless binary decoding operations;
generating a reconstructed target feature representation based at least in part on the decompressed target feature representation and a machine-learned decoding model;
determining a localized state of the source object within the target representation of the environment based at least in part on one or more comparisons of the source feature representation to the reconstructed target feature representation;
determining a loss based at least in part on one or more comparisons of the localized state of the source object to the ground-truth state of the source object; and
adjusting one or more parameters of the one or more machine-learned compression models based at least in part on the loss.

10. The computing system of claim 9 , wherein the generating the compressed target feature representation of the target feature representation based at least in part on the one or more machine-learned compression models comprises:
generating an encoded target feature representation based at least in part the target feature representation and a machine-learned encoding model;
generating the compressed target feature representation based at least in part on use of one or more lossless binary encoding operations on the encoded target feature representation; and
wherein adjusting the one or more parame

Assignees

• Uatc Llc

Inventors

• Urtasun Raquel
• Wei Xinkai
• Barsan Ioan Andrei
• Covarrubias Julieta Martinez
• Wang Shenlong

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/08

  Learning methods · CPC title

• G06N3/0495

  Quantised networks; Sparse networks; Compressed networks · CPC title

• G06N3/09

  Supervised learning · CPC title