Using maps comprising covariances in multi-resolution voxels

Patent metadata

Field	Value
Publication number	US-12293541-B2
Application number	US-202217897491-A
Country	US
Kind code	B2
Filing date	Aug 29, 2022
Priority date	Dec 20, 2019
Publication date	May 6, 2025
Grant date	May 6, 2025

Abstract

Techniques for representing a scene or map based on statistical data of captured environmental data are discussed herein. In some cases, the data (such as covariance data, mean data, or the like) may be stored as a multi-resolution voxel space that includes a plurality of semantic layers. In some instances, individual semantic layers may include multiple voxel grids having differing resolutions. Multiple multi-resolution voxel spaces may be merged to generate combined scenes based on detected voxel covariances at one or more resolutions.

First claim

What is claimed is:

1. One or more non-transitory computer-readable media storing instructions that, when executed, cause one or more processors to perform operations comprising:
associating map data with a first space;
receiving sensor data from a sensor of a vehicle in an environment;
associating the sensor data with a second space;
determining, based at least in part on the first space and the second space, aggregated data;
determining a statistical value associated with a portion of the aggregated data, wherein the statistical value comprises a weighted covariance matrix associated with a voxel of the second space;
determining, based at least in part on the statistical value, a difference between the first space and the second space; and
determining a location of the vehicle in the environment based at least in part on the difference.

2. The one or more non-transitory computer-readable media of claim 1 , wherein determining the aggregated data comprises:
identifying, for a first portion of the first space, a set of portions of the second space that have a centroid within a specified distance of a centroid of the first portion; and
determining a second portion of the set of portions, the second portion having a centroid nearest to the centroid of the first portion.

3. The one or more non-transitory computer-readable media of claim 1 , wherein:
the first space is defined by a plurality of resolutions,
the second space is defined by the plurality of resolutions, and
determining the aggregated data further comprises associating data of a first portion having a resolution of the first space with a second portion of the second space having a same resolution.

4. The one or more non-transitory computer-readable media of claim 1 , wherein the first space is a first voxel space and the second space is a second voxel space.

5. The one or more non-transitory computer-readable media of claim 1 , wherein determining the difference comprises:
determining a smallest eigenvector of the statistical value;
determining, based at least in part on the smallest eigenvector, a residual; and
determining, as the difference, one or more of a rotation or a translation between the first space and the second space based at least in part on the residual.

6. The one or more non-transitory computer-readable media of claim 5 , the operations further comprising determining an uncertainty associated with the difference based at least in part on modeling a distribution.

7. The one or more non-transitory computer-readable media of claim 1 , wherein the difference comprises a difference in one or more of a position or orientation between the first space and second space.

8. The one or more non-transitory computer-readable media of claim 1 , wherein the vehicle is an autonomous vehicle, the operations further comprising:
controlling the autonomous vehicle based at least in part on the location of the autonomous vehicle in the environment.

9. The one or more non-transitory computer-readable media of claim 1 , the operations further comprising:
performing principal component analysis on the weighted covariance matrix; and
determining a smallest eigenvector of the principal component analysis, wherein determining the difference is further based on the smallest eigenvector.

10. A method comprising:
associating map data with a first space;
receiving sensor data from a sensor of a vehicle in an environment;
associating the sensor data with a second space;
determining, based at least in part on the first space and the second space, aggregated data;
determining a statistical value associated with a portion of the aggregated data, wherein the statistical value comprises a weighted covariance matrix associated with a voxel of the second space;
determining, based at least in part on the statistical value, a difference between the first space and the second space; and
determining a location of the vehicle in the environment based at least in part on the difference.

11. The method of claim 10 , wherein determining the aggregated data comprises:
identifying, for a first portion of the first space, a set of portions of the second space that have a centroid within a specified distance of a centroid of the first portion; and
determining a second portion of the set of portions, the second portion having a centroid nearest to the centroid of the first portion.

12. The method of claim 10 , wherein the first space is defined by a plurality of resolutions;
the second space is defined by the plurality of resolutions; and
determining the aggregated data further comprises associating data of a first portion having a resolution of the first space with a second portion of the second space having the resolution.

13. The method of claim 10 , wherein determining the difference comprises:
determining a smallest eigenvector of the statistical value;
determining, based at least in part on the smallest eigenvector, a residual; and
determining, as the difference, one or more of a rotation or a translation between the first space and the second space based at least in part on the residual.

14. The method of claim 10 , the method further comprising:
performing principal component analysis on the weighted covariance matrix; and
determining a smallest eigenvector of the principal component analysis, wherein determining the difference is further based on the smallest eigenvector.

15. A system comprising:
one or more processors; and
one or more non-transitory computer readable media storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the system to perform operations comprising:
associating map data with a first space;
receiving sensor data from a vehicle in an environment;
associating the sensor data with a second space;
determining, based at least in part on the first space and the second space, aggregated data;
determining a statistical value associated with a portion of the aggregated data, wherein the statistical value comprises a weighted covariance matrix associated with a voxel of the second space;
determining, based at least in part on the statistical value, a difference between the first space and the second space; and
determining a location of the vehicle in the environment based at least in part on the difference.

16. The system of claim 15 , wherein determining the aggregated data comprises:
identifying, for a first portion of the first space, a set of portions of the second space that are within a specified distance of the first portion;
determining a second portion of the set of portions, the second portion being nearest to the first portion; and
determining a weighted average of a covariance associated with the first portion and a covariance associated with the second portion of the set of portions.

17. The system of claim 15 , wherein the first space is defined by a plurality of resolutions;
the second space is defined by the plurality of resolutions; and
determining the aggregated data further comprises associating data of a first portion having a resolution of the first space with a second portion having the resolution of the second space.

18. The system of claim 15 , wherein determining the difference comprises:
determining a smallest eigenvector of the statistical value; and
determining one or more of a rotation or a translation between the first space and the second space based at least in part on the smallest eigenvector.

19. The system of claim 15

Assignees

• Zoox Inc

Inventors

• Bosse Michael Carsten
• Blaes Patrick
• Adams Derek
• Rebsamen Brice

Classifications

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2135

  based on approximation criteria, e.g. principal component analysis · CPC title

• G06T7/74

  involving reference images or patches · CPC title