Capture and display of point clouds using augmented reality device

Patent metadata

Field	Value
Publication number	US-12561922-B2
Application number	US-202418658567-A
Country	US
Kind code	B2
Filing date	May 8, 2024
Priority date	May 8, 2024
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Techniques for capturing and displaying point clouds using an AR device are described. A GNSS point of the AR device is determined based on received satellite signals. A depth image is captured while the AR device is positioned at the GNSS point. The depth image is projected into 3D space to obtain a point cloud having a set of points. An accuracy of the GNSS point is determined. In response to determining that the accuracy of the GNSS point is below a threshold, the GNSS point is determined to be a low-accuracy GNSS point, points from the set of points that are outside of a zone surrounding a previously determined high-accuracy GNSS point are removed, and remaining points from the set of points that are inside the zone are displayed on a display of the AR device.

First claim

What is claimed is:

1 . A method of capturing point clouds using an augmented reality (AR) device, the method comprising:
determining a global navigation satellite systems (GNSS) point of the AR device based on received satellite signals; capturing a depth image while the AR device is positioned at the GNSS point; projecting the depth image into 3D space to obtain a point cloud having a set of points; determining an accuracy of the GNSS point; and in response to determining that the accuracy of the GNSS point is below a threshold:
determining that the GNSS point is a low-accuracy GNSS point;
removing points from the set of points that are outside of a zone surrounding a previously determined high-accuracy GNSS point; and
displaying remaining points from the set of points that are inside the zone on a display of the AR device.

2 . The method of claim 1 , wherein the depth image is projected into 3D space using intrinsic parameters of a camera of the AR device.

3 . The method of claim 1 , further comprising:
determining a second GNSS point of the AR device based on the received satellite signals; capturing a second depth image while the AR device is positioned at the second GNSS point; projecting the second depth image into 3D space to obtain a second point cloud having a second set of points; determining an accuracy of the second GNSS point; in response to determining that the accuracy of the second GNSS point is above the threshold:
determining that the second GNSS point is a high-accuracy GNSS point;
updating the zone to surround the second GNSS point; and
displaying each of the second set of points on the display.

4 . The method of claim 3 , further comprising:
in response to determining that the accuracy of the GNSS point is below the threshold, storing the remaining points from the set of points in a database in an accumulated point cloud; in response to determining that the accuracy of the second GNSS point is above the threshold, storing each of the second set of points in the database in the accumulated point cloud; and displaying the accumulated point cloud on the display including the remaining points from the set of points and each of the second set of points.

5 . The method of claim 3 , further comprising:
transforming the point cloud into an AR reference frame using an AR transformation matrix; and in response to determining that the accuracy of the second GNSS point is above the threshold, using the GNSS point to correlate the AR reference frame with a geospatial reference frame.

6 . The method of claim 1 , further comprising:
displaying a virtual representation of the zone on the display.

7 . The method of claim 1 , wherein the AR device comprises a camera component including a camera and a sensor component including a GNSS receiver, and wherein the camera component is separable from and configured to removably attach to the sensor component.

8 . A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations for capturing point clouds using an augmented reality (AR) device, the operations comprising:
determining a global navigation satellite systems (GNSS) point of the AR device based on received satellite signals; capturing a depth image while the AR device is positioned at the GNSS point; projecting the depth image into 3D space to obtain a point cloud having a set of points; determining an accuracy of the GNSS point; and in response to determining that the accuracy of the GNSS point is below a threshold:
determining that the GNSS point is a low-accuracy GNSS point;
removing points from the set of points that are outside of a zone surrounding a previously determined high-accuracy GNSS point; and
displaying remaining points from the set of points that are inside the zone on a display of the AR device.

9 . The non-transitory computer-readable medium of claim 8 , wherein the depth image is projected into 3D space using intrinsic parameters of a camera of the AR device.

10 . The non-transitory computer-readable medium of claim 8 , wherein the operations further comprise:
determining a second GNSS point of the AR device based on the received satellite signals; capturing a second depth image while the AR device is positioned at the second GNSS point; projecting the second depth image into 3D space to obtain a second point cloud having a second set of points; determining an accuracy of the second GNSS point; and in response to determining that the accuracy of the second GNSS point is above the threshold:
determining that the second GNSS point is a high-accuracy GNSS point;
updating the zone to surround the second GNSS point; and
displaying each of the second set of points on the display.

11 . The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:
in response to determining that the accuracy of the GNSS point is below the threshold, storing the remaining points from the set of points in a database in an accumulated point cloud; in response to determining that the accuracy of the second GNSS point is above the threshold, storing each of the second set of points in the database in the accumulated point cloud; and displaying the accumulated point cloud on the display including the remaining points from the set of points and each of the second set of points.

12 . The non-transitory computer-readable medium of claim 10 , wherein the operations further comprise:
transforming the point cloud into an AR reference frame using an AR transformation matrix; and in response to determining that the accuracy of the second GNSS point is above the threshold, using the GNSS point to correlate the AR reference frame with a geospatial reference frame.

13 . The non-transitory computer-readable medium of claim 8 , wherein the operations further comprise:
displaying a virtual representation of the zone on the display.

14 . The non-transitory computer-readable medium of claim 8 , wherein the AR device comprises a camera component including a camera and a sensor component including a GNSS receiver, and wherein the camera component is separable from and configured to removably attach to the sensor component.

15 . A system comprising:
one or more processors; and a non-transitory computer-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform operations for capturing point clouds using an augmented reality (AR) device, the operations comprising:
determining a global navigation satellite systems (GNSS) point of the AR device based on received satellite signals;
capturing a depth image while the AR device is positioned at the GNSS point;
projecting the depth image into 3D space to obtain a point cloud having a set of points;
determining an accuracy of the GNSS point; and
in response to determining that the accuracy of the GNSS point is below a threshold:
determining that the GNSS point is a low-accuracy GNSS point;
removing points from the set of points that are outside of a zone surrounding a previously determined high-accuracy GNSS point; and
displaying remaining points from the set of points that are inside the zone on a display of the AR device.

16 . The system of claim 15 , wherein the depth image is projected into 3D space using intrinsic parameters of a camera of the AR device.

17 . The system of claim 15 , wherein the operations further com

Assignees

• Trimble Inc

Inventors

• Bursill Adam
• Mcloughlin Callum

Classifications

• G01S19/14 Primary

  specially adapted for specific applications · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title