Automatic selection of delivery zones using survey flight 3D scene reconstructions

Patent metadata

Field	Value
Publication number	US-12399508-B2
Application number	US-202218056710-A
Country	US
Kind code	B2
Filing date	Nov 17, 2022
Priority date	Nov 17, 2022
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

A method includes navigating, by an uncrewed aerial vehicle (UAV), to a delivery location in an environment. The method also includes capturing, by at least one sensor on the UAV, sensor data representative of the delivery location. The method further includes determining, based on the sensor data representative of the delivery location, a segmented point cloud. The segmented point cloud defines a point cloud of the delivery location segmented into a plurality of point cloud areas with corresponding semantic classifications. The method additionally includes determining, based on the segmented point cloud, at least one delivery point in the delivery location. The at least one delivery point in the delivery location satisfies at least one condition indicating that a descent path above the at least one delivery point represented in the point cloud is at least a particular lateral distance away from point cloud areas with corresponding semantic classifications indicative of an obstacle at the delivery location. The method also includes transmitting, by the UAV, the at least one delivery point to a server device.

First claim

What is claimed is:

1. A method comprising:
navigating, by an uncrewed aerial vehicle (UAV), to a delivery location in an environment;
capturing, by at least one sensor on the UAV, sensor data representative of the delivery location;
determining, based on the sensor data representative of the delivery location, a segmented point cloud, wherein the segmented point cloud defines a point cloud of the delivery location segmented into a plurality of point cloud areas with corresponding semantic classifications;
determining, based on the segmented point cloud, at least one delivery point in the delivery location, wherein the at least one delivery point in the delivery location satisfies at least one condition, each condition indicating that a descent path through a cylinder centered above the at least one delivery point with a radius of a particular lateral distance does not intersect with point cloud areas with corresponding semantic classifications indicative of an obstacle at the delivery location; and
transmitting, by the UAV, the at least one delivery point to a server device.

2. The method of claim 1 , wherein the at least one condition comprises a plurality of conditions, each of which is associated with a different semantic classification.

3. The method of claim 2 , wherein each of the plurality of conditions is further associated with a different particular lateral distance away from point cloud areas with a respective semantic classification.

4. The method of claim 1 , wherein determining, based on the segmented point cloud, the at least one delivery point in the delivery location comprises selecting a delivery point from a plurality of candidate delivery points evenly spaced in a grid pattern in the environment.

5. The method of claim 1 , wherein the at least one delivery point in the delivery location satisfies an additional condition indicating that the descent path above the at least one delivery point represented in the point cloud is at least an additional particular lateral distance away from point cloud areas with corresponding semantic classifications indicative of an obstacle at the delivery location, wherein the additional particular lateral distance is greater than the particular lateral distance and enables landing of the UAV at the delivery location.

6. The method of claim 1 , wherein the at least one condition comprises a condition indicating that the descent path above the at least one delivery point represented in the point cloud is at least a particular lateral distance away from point cloud areas with corresponding semantic classifications indicative of a building at the delivery location.

7. The method of claim 1 , wherein the at least one condition comprises a first condition and a second condition, wherein the first condition indicates that the descent path is at least a first lateral distance away from point cloud areas with corresponding semantic classifications indicative of a building of a first height, wherein the second condition indicates that the descent path is at least a second lateral distance away from point cloud areas with corresponding semantic classifications indicative of a building of a second height, wherein the first height is greater than the second height and the first lateral distance is greater than the second lateral distance.

8. The method of claim 1 , wherein the descent path is from a ground surface at the at least one delivery point to a predetermined altitude above the at least one delivery point, wherein the predetermined altitude is associated with where the UAV captured the sensor data.

9. The method of claim 1 , wherein the sensor data comprises two-dimensional representations of the delivery location, wherein the point cloud is a three-dimensional representation of the delivery location.

10. The method of claim 1 , wherein determining, based on the sensor data of the delivery location, a segmented point cloud is based on applying at least one pre-trained machine learning model to the sensor data of the delivery location.

11. The method of claim 1 , further comprising:
selecting a delivery point from the at least one delivery point based on determining that the delivery point is at a particular location relative to a building.

12. The method of claim 1 , wherein the method further comprises:
transmitting a request to the server device for a delivery point at the delivery location; and
receiving, from the server device, a response comprising the delivery point.

13. The method of claim 1 , further comprising:
capturing, by the UAV, one or more additional images of the delivery location;
verifying, based on the one or more additional images of the delivery location, whether a selected delivery point of the at least one delivery point satisfies the at least one condition; and
based on verifying that the selected delivery point does satisfy the at least one condition, descending to be a particular altitude above the selected delivery point.

14. The method of claim 1 , wherein the method further comprises:
transmitting a request to the server device for a flight trajectory to the at least one delivery point at the delivery location;
receiving, from the server device, a response comprising the flight trajectory to the at least one delivery point;
navigating, by the UAV, along the flight trajectory to the delivery point;
while navigating along the flight trajectory to the delivery point, determining one or more obstacles along the flight trajectory; and
navigating to avoid the one or more obstacles in the flight trajectory.

15. The method of claim 1 , wherein determining, based on the segmented point cloud, at least one delivery point in the delivery location comprises selecting the at least one delivery point to be on a surface represented in the point cloud with a corresponding semantic classification to which delivery is permitted.

16. The method of claim 1 , wherein capturing, by the at least one sensor on the UAV, the sensor data of the delivery location comprises:
navigating the UAV to a plurality of locations at the delivery location; and
capturing an image at each of the plurality of locations.

17. The method of claim 16 , wherein navigating the UAV to a plurality of locations at the delivery location comprises navigating in a lawnmower pattern.

18. The method of claim 1 , wherein the capturing, by the at least one sensor on the UAV, sensor data of the delivery location comprises capturing one or more images of the delivery location while descending over the delivery location.

19. The method of claim 1 , wherein the sensor on the UAV is a camera or a LIDAR sensor.

20. The method of claim 1 , wherein the semantic classifications indicative of an obstacle comprise semantic classifications corresponding to an unacceptable delivery surface and semantic classifications corresponding to an object exceeding a threshold height.

21. An uncrewed aerial vehicle (UAV), comprising:
at least one sensor; and
a control system configured to:
navigate, by the UAV, to a delivery location in an environment;
capture, by the at least one sensor on the UAV, sensor data representative of the delivery location;
determine, based on the sensor data representative of the delivery location, a segmented point cloud, wherein the segmented point cloud defines a point cloud of the delivery location segmented into a plurality of point cloud areas with corresponding semantic classifications;
determine, based on the segmented point cloud, at least o

Assignees

• Wing Aviation Llc

Inventors

• Shoeb Ali

Classifications

• G05D2111/10

  Optical signals · CPC title

• G05D2107/17

  Spaces with priority for humans, e.g. populated areas, pedestrian ways, parks or beaches · CPC title

• G05D2109/254

  Flying platforms, e.g. multicopters · CPC title

• G05D1/665

  Releasing payloads, e.g. parachutes · CPC title

• G05D1/689

  Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672) · CPC title