Determining a three-dimensional model of a scan target

Patent metadata

Field	Value
Publication number	US-12189389-B2
Application number	US-202318520072-A
Country	US
Kind code	B2
Filing date	Nov 27, 2023
Priority date	Feb 13, 2020
Publication date	Jan 7, 2025
Grant date	Jan 7, 2025

Abstract

In some examples, one or more processors of an unmanned aerial vehicle (UAV), control a propulsion mechanism of the UAV to cause the UAV to navigate to a plurality of positions in relation to a scan target. Using one or more image sensors of the UAV, a first image of the scan target is captured from a first position of the plurality of positions, and a second image of the scan target is captured from a second position of the plurality of positions. A disparity is determined between the first image captured at the first position and the second image captured at the second position. A three-dimensional model corresponding to the scan target is determined based in part on the disparity determined between the first image and the second image.

First claim

What is claimed:

1. An unmanned aerial vehicle (UAV) comprising:
one or more image sensors;
a propulsion mechanism; and
one or more processors configured by executable instructions to at least:
control the propulsion mechanism to cause the UAV to navigate to a plurality of positions in relation to a scan target;
capture, using the one or more image sensors, a first image of the scan target from a first position of the plurality of positions, and a second image of the scan target from a second position of the plurality of positions;
determine a disparity between the first image captured at the first position and the second image captured at the second position; and
determine a three-dimensional (3D) model corresponding to the scan target based in part on the disparity determined between the first image and the second image.

2. The UAV as recited in claim 1 , the executable instructions further configuring the one or more processors to use distance information determined based at least on the disparity determined between the first image and the second image for determining respective locations in 3D space of one or more points of the 3D model, the one or more points representative of a surface of the scan target.

3. The UAV as recited in claim 1 , the executable instructions further configuring the one or more processors to:
based at least in part on determining the 3D model, determine a scan plan for scanning at least a portion of the scan target, the scan plan including a plurality of poses for the UAV to assume to capture images of the target, each pose corresponding to a point of a plurality of points included in the 3D model; and
control the propulsion mechanism of the UAV to cause the UAV to fly to the plurality of poses included in the scan plan.

4. The UAV as recited in claim 1 , wherein the 3D model is determined based at least in part on at least one of:
multi-view stereo analysis of the first image and the second image; or
performing Truncated Signed Distance Function (TSDF) fusion for the first image and the second image.

5. The UAV as recited in claim 1 , the executable instructions further configuring the one or more processors to navigate the UAV in relation to the scan target based at least in part on receiving an indication of the scan target, the indication of the scan target including at least one of:
an indication of a 2D shape forming a boundary within which at least a portion the scan target is located;
an indication of a shape corresponding to a surface of the scan target; or
an indication of a volume within which at least a portion of the scan target is located.

6. The UAV as recited in claim 1 , the executable instructions further configuring the one or more processors to navigate the UAV in relation to the scan target based at least in part on receiving an indication of the scan target via a user interface of a computing device in communication with the UAV while the UAV is in flight, wherein the indication of the scan target is based at least in part on an image of the scan target transmitted by the UAV to the computing device.

7. The UAV as recited in claim 1 , the executable instructions further configuring the one or more processors to:
capture a third image at a third position, wherein the first position, the second position, and the third position:
all correspond to the UAV traveling along a straight path in 3D space; or
all correspond to the UAV traveling along a curved path in 3D space; and

additionally determine the 3D model corresponding to the scan target based in part on at least one of:
a disparity determined between the first image and the third image, or
a disparity determined between the second image and the third image.

8. A method comprising:
controlling, by one or more processors of an unmanned aerial vehicle (UAV), a propulsion mechanism of the UAV to cause the UAV to navigate to a plurality of positions in relation to a scan target;
capturing, by the one or more processors of the UAV, using one or more image sensors of the UAV, a first image of the scan target from a first position of the plurality of positions, and a second image of the scan target from a second position of the plurality of positions;
determining, by the one or more processors of the UAV, a disparity between the first image captured at the first position and the second image captured at the second position; and
determining, by the one or more processors of the UAV, a three-dimensional (3D) model corresponding to the scan target based in part on the disparity determined between the first image and the second image.

9. The method as recited in claim 8 , further comprising:
determining, based at least on the disparity determined between the first image and the second image, distance information indicative of distance to one or more points of a surface of the scan target.

10. The method as recited in claim 9 , further comprising:
using the distance information determined based at least on the disparity determined between the first image and the second image for determining respective locations in 3D space of one or more points of the 3D model, the one or more points representative of the surface of the scan target.

11. The method as recited in claim 8 , further comprising:
based at least in part on determining the 3D model, determining a scan plan for scanning at least a portion of the scan target, the scan plan including a plurality of poses for the UAV to assume to capture images of the target, each pose corresponding to a point of a plurality of points included in the 3D model; and
controlling the propulsion mechanism of the UAV to cause the UAV to fly to the plurality of poses included in the scan plan.

12. The method as recited in claim 8 , wherein the 3D model is determined based at least in part on at least one of:
multi-view stereo analysis of the first image and the second image; or
performing Truncated Signed Distance Function (TSDF) fusion for the first image and the second image.

13. The method as recited in claim 8 , further comprising:
navigating the UAV in relation to the scan target based at least in part on receiving an indication of the scan target, the indication of the scan target including at least one of:
an indication of a 2D shape forming a boundary within which at least a portion the scan target is located;
an indication of a shape corresponding to a surface of the scan target; or
an indication of a volume within which at least a portion of the scan target is located.

14. The method as recited in claim 8 , further comprising:
navigating the UAV in relation to the scan target based at least in part on receiving an indication of the scan target via a user interface of a computing device in communication with the UAV while the UAV is in flight, wherein the indication of the scan target is based at least in part on an image of the scan target transmitted by the UAV to the computing device.

15. The method as recited in claim 8 , further comprising:
capturing a third image at a third position, wherein the first position, the second position, and the third position:
all correspond to the UAV traveling along a straight path in 3D space; or
all correspond to the UAV traveling along a curved path in 3D space; and

additionally determining the 3D model corresponding to the scan target based in part on at least one of:
a disparity determined between the first image and the third image, or
a disparity determined between the second image and the third image.

16. An aerial vehicle comprising
one or more image s

Assignees

• Skydio Inc

Inventors

• Henry Peter
• Zhu Jack
• Richman Brian
• Zheng Harrison
• Martirosyan Hayk
• Donahoe Matthew
• Bachrach Abraham
• Bry Adam
• Kennedy Ryan David
• Mondal Himel
• Delepine Quentin Allen Wah Yen

Classifications

• B64D31/06

  actuated automatically · CPC title

• B64D47/08

  Arrangements of cameras · CPC title

• B64C39/024

  of the remote controlled vehicle type, i.e. RPV · CPC title

• B64U10/13

  Flying platforms · CPC title

• B64U2201/20

  Remote controls · CPC title