Fitness and sports applications for an autonomous unmanned aerial vehicle

Patent metadata

Field	Value
Publication number	US-12443186-B2
Application number	US-202217874524-A
Country	US
Kind code	B2
Filing date	Jul 27, 2022
Priority date	Jun 12, 2018
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

Sports and fitness applications for an autonomous unmanned aerial vehicle (UAV) are described. In an example embodiment, a UAV can be configured to track a human subject using perception inputs from one or more onboard sensors. The perception inputs can be utilized to generate values for various performance metrics associated with the activity of the human subject. In some embodiments, the perception inputs can be utilized to autonomously maneuver the UAV to lead the human subject to satisfy a performance goal. The UAV can also be configured to autonomously capture images of a sporting event and/or make rule determinations while officiating a sporting event.

First claim

What is claimed is:

1. A method for facilitating fitness training using an unmanned aerial vehicle (UAV), the method comprising:
receiving, by a computer system onboard the UAV, images of a physical environment captured by one or more image capture devices associated with the UAV, the UAV in autonomous flight through the physical environment in proximity to a human subject;
processing, by the computer system, the received images to detect and track a motion of the human subject through the physical environment;
analyzing, by the computer system, the motion of the human subject based on the tracking;
determining, by the computer system, based on the analysis, a value for a performance metric associated with the motion of the human subject;
generating, by the computer system, a graphical element indicative of the performance metric;
generating, by the computer system, an interactive control element configured to respond to user interactions to control an angle at which the UAV captures the human subject while in autonomous flight through the physical environment;
generating, by the computer system, a visual output that includes a composite of at least some of the images of the physical environment overlaid with augmentations of the graphical element indicative of the performance metric and the interactive control element, wherein a size and placement of the augmentations is based, at least in part, on a size and geometry of the human subject; and
transmitting the visual output for display on a mobile device.

2. The method of claim 1 , further comprising:
causing display, by the computer system, of the visual output at the mobile device.

3. The method of claim 2 , wherein the mobile device is communicatively coupled to the UAV via a wireless communication link and wherein the visual output is displayed at the mobile device in real-time as the UAV is in autonomous flight and tracking the human subject.

4. The method of claim 1 , wherein the graphical element includes
a graphical representation of a trajectory of the human subject; or
a graphical representation of a skeletal structure of the human subject.

5. The method of claim 1 , wherein the performance metric includes any of speed, total run time, lap time, gait, pace, or elevation gain.

6. The method of claim 1 , wherein the human subject is any of a runner, a swimmer, a bicyclist, a skier, or a snowboarder.

7. An unmanned aerial vehicle (UAV) configured for facilitating fitness training, the UAV comprising:
a propulsion system;
one or more image capture devices; and
an onboard computer system communicatively coupled to the propulsion system and the one or more image capture devices, the computer system configured to:
receive images of a physical environment captured by the one or more image capture devices while the UAV is in autonomous flight through the physical environment in proximity to a human subject;
process the images to detect and track motion of the human subject through the physical environment;
analyze the motion of the human subject based on the tracking to determine a value for a performance metric;
generate a graphical element indicative of the performance metric;
generate an interactive control element configured to respond to user interactions to control an angle at which the UAV captures the human subject while in autonomous flight through the physical environment;
generate a visual output that includes a composite of at least some of the images of the physical environment overlaid with augmentations of the graphical element indicative of the performance metric and the interactive control element, wherein a size and placement of the augmentations is based, at least in part, on a size and geometry of the human subject;
and
cause display of the visual output.

8. The UAV of claim 7 , wherein to cause display of the visual output, the computer system is configured to cause display of the visual output at a mobile device.

9. The UAV of claim 8 , wherein the mobile device is communicatively coupled to the UAV via a wireless communication link and wherein the visual output is displayed at the mobile device in real-time as the UAV is in autonomous flight and tracking the human subject.

10. The UAV of claim 7 , wherein the graphical element includes a graphical representation of a trajectory of the human subject.

11. The UAV of claim 7 , wherein the performance metric includes any of speed, total run time, lap time, gait, pace, or elevation gain.

12. The UAV of claim 7 , wherein the human subject is any of a runner, a swimmer, a bicyclist, a skier, or a snowboarder.

13. An apparatus comprising
one or more computer-readable media; and
program instructions stored on the one or more computer-readable storage media that, when executed by one or more processors onboard an aerial vehicle, direct the one or more processors to at least:
receive images of a physical environment captured by the one or more image capture devices while the UAV is in autonomous flight through the physical environment in proximity to a human subject;
process the images to detect and track motion of the human subject through the physical environment;
analyze the motion of the human subject based on the tracking to generate a value for a performance metric;
generate a graphical element indicative of the performance metric;
generate an interactive control element configured to respond to user interactions to control an angle at which the UAV captures the human subject while in autonomous flight through the physical environment;
generate a visual output that includes a composite of at least some of the images of the physical environment overlaid with augmentations of the graphical element indicative of the performance metric and the interactive control element, wherein a size and placement of the augmentations is based, at least in part, on a size and geometry of the human subject; and

cause display of the visual output.

14. The apparatus of claim 13 , wherein to cause display of the visual output, the one or more processors are configured to cause display of the visual output at a mobile device.

15. The apparatus of claim 14 , wherein the mobile device is communicatively coupled to the UAV via a wireless communication link and wherein the visual output is displayed at the mobile device in real-time as the UAV is in autonomous flight and tracking the human subject.

16. The apparatus of claim 13 , wherein the graphical element includes a graphical representation of a trajectory of the human subject.

17. The apparatus of claim 13 , wherein the performance metric includes any of speed, total run time, lap time, gait, pace, or elevation gain.

18. The method of claim 1 , wherein the graphical element includes a graphical representation of a skeletal structure of the human subject.

19. The UAV of claim 7 , wherein the graphical element includes a graphical representation of a skeletal structure of the human subject.

20. The apparatus of claim 13 , wherein the graphical element includes a graphical representation of a skeletal structure of the human subject.

Assignees

• Skydio Inc

Inventors

• Bachrach Abraham Galton
• Bry Adam Parker
• Donahoe Matthew Joseph
• Martirosyan Hayk
• Moss Tom

Classifications

• G05D1/689

  Pointing payloads towards fixed or moving targets (positioning towed, pushed or suspended implements G05D1/672) · CPC title

• B64U10/14

  with four distinct rotor axes, e.g. quadcopters · CPC title

• B64U20/87

  Mounting of imaging devices, e.g. mounting of gimbals · CPC title

• B64U10/25

  Fixed-wing aircraft (VTOL aircraft B64U10/20) · CPC title

• G06V20/17

  taken from planes or by drones · CPC title