Distributed computing network to perform simultaneous localization and mapping

Patent metadata

Field	Value
Publication number	US-12067765-B2
Application number	US-202117196761-A
Country	US
Kind code	B2
Filing date	Mar 9, 2021
Priority date	Mar 9, 2021
Publication date	Aug 20, 2024
Grant date	Aug 20, 2024

Abstract

An apparatus includes a processing node of a distributed computing platform. The processing node communicates with other processing nodes over one or more networks. The processing node may receive frames of point clouds at a processing node of a distributed computing platform, determine a subset of the frames as key frames based at least in part on distances travelled between captures of the respective frames, and allocate tasks of processing the key frames to processing subnodes based at least in part on estimated processing demands of the key frames and processing capabilities of each of the processing subnodes.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
receiving frames of point clouds at a processing node of a distributed computing platform;
determining, from the frames, first frames in which first distances among the first frames exceed a given threshold;
determining, as key frames, the first frames and a second subset of second frames, wherein:
each of the second frames comprise at least a common feature as present in the first frames, the common feature in the second frames being at a higher resolution compared to that in the first frames; and

allocating tasks of processing the key frames to processing subnodes based at least in part on estimated processing demands of the key frames and processing capabilities of each of the processing subnodes.

2. The computer-implemented method of claim 1 , wherein the allocating of tasks comprises:
allocating key frames having a first range of timestamps to a first processing subnode; and
allocating second key frames having a second range of timestamps to a second processing subnode.

3. The computer-implemented method of claim 1 , wherein the determination of the key frames comprises determining a frame to be a key frame in response determining that a position corresponding to a capture of the frame deviates from positions corresponding to captures of remaining frames by more than a threshold distance.

4. The computer-implemented method of claim 1 , wherein the tasks of processing the key frames include determining loop closure constraints based on common features identified in the key frames, the loop closure constraints indicating that a pair of the key frames were captured at locations within a threshold distance of each other.

5. The computer-implemented method of claim 4 , wherein, following the determination of the loop closure constraints, the loop closure constraints are transmitted to a downstream node, the downstream node aggregating the loop closure constraints and determining positions at which each of the key frames were captured based on the aggregated loop closure constraints.

6. The computer-implemented method of claim 5 , wherein, the downstream node further determines locations of the common features.

7. The computer-implemented method of claim 6 , wherein, following the determination of the positions at which each of the key frames were captured, the determined positions are transmitted to the processing subnodes, the processing subnodes generating partial static maps from the respective allocated keyframes based on the transmitted determined positions.

8. The computer-implemented method of claim 7 , wherein the generated partial static maps are transmitted to the downstream node, the downstream node aggregating the generated partial static maps to generate an integrated static map.

9. The computer-implemented method of claim 1 , wherein the determination of key frames is based on an identification of features and contexts of the features in the frames using a trained machine learning model, and training datasets used to train the machine learning model comprise:
a first dataset including features and contexts that would result in a frame being determined to be a key frame; and
a second dataset including second features and second contexts that would result in a frame being determined to be a non-key frame.

10. The computer-implemented method of claim 1 , wherein the processing demands of the key frames are determined at least in part by respective degrees of reliability of position and attitude estimations obtained from the key frames.

11. The computer-implemented method of claim 1 , wherein the allocating of tasks comprises:
allocating a task to a processing subnode if frames corresponding to a same segment as the task are cached in the processing subnode.

12. The computer-implemented method of claim 1 , wherein the key frames exclude any frames besides the first frames, the second frames, and the third frames.

13. The computer-implemented method of claim 1 , further comprising:
determining, as key frames, a third subset of third frames, wherein:
each of the third frames comprise the common feature or a different common feature as present in the first frames, the common feature or the different common feature in the third frames being from a different perspective compared to that in the first frames.

14. The computer-implemented method of claim 1 , wherein the allocating of tasks of processing the key frames to processing subnodes comprises dividing the processing of the key frames among the processing subnodes based on a proportion of the key frames that have potentially unreliable position and attitude estimates.

15. The computer-implemented method of claim 1 , wherein at least a subset of the processing subnodes determine one or more additional loop closure constraints based on odometry data.

16. The computer-implemented method of claim 1 , wherein at least a subset of the processing subnodes:
attempt a point cloud registration on point clouds captured at one or more times corresponding to a potential loop closure;
determine a degree of alignment of resulting registered point clouds; and
if the degree of alignment of any resulting registered point clouds fail to satisfy a threshold degree of alignment, keeping the point clouds separate.

17. The computer-implemented method of claim 1 , wherein at least a subset of the processing subnodes:
receive, from downstream subnodes, one or more potential poses of a vehicle at different times of capture of the point clouds; and
augment the potential poses with odometry data generated by an inertial measurement unit (IMU) and a Global Positioning System (GPS) sensor.

18. An apparatus comprising:
a processing node of a distributed computing platform, the processing node being one of a plurality of processing nodes and being configured to communicate with other ones of the plurality of processing nodes over one or more networks; the processing node being further configured to:
receive frames of point clouds at a processing node of a distributed computing platform;
determine, from the frames, first frames in which first distances among the first frames exceed a given threshold;
determine, as key frames, the first frames and a second subset of second frames, wherein:
each of the second frames comprise at least a common feature as present in the first frames, the common feature in the second frames being at a higher resolution compared to that in the first frames; and

allocate tasks of processing the key frames to processing subnodes based at least in part on estimated processing demands of the key frames and processing capabilities of each of the processing subnodes.

19. The apparatus of claim 18 , wherein the allocating of tasks comprises:
allocating key frames having a first range of timestamps to a first processing subnode; and
allocating second key frames having a second range of timestamps to a second processing subnode.

20. The apparatus of claim 18 , wherein the determination of the key frames comprises determining a frame to be a key frame in response determining that a position corresponding to a capture of the frame deviates from positions corresponding to captures of remaining frames by more than a threshold distance.

Assignees

• Pony Ai Inc

Inventors

• Yang Mengda
• Shi Ruimeng

Classifications

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06T7/73

  using feature-based methods · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title