Fitness and sports applications for an autonomous unmanned aerial vehicle

Patent metadata

Field	Value
Publication number	US-2019377345-A1
Application number	US-201916439504-A
Country	US
Kind code	A1
Filing date	Jun 12, 2019
Priority date	Jun 12, 2018
Publication date	Dec 12, 2019
Grant date	—

Abstract

Sports and fitness applications for an autonomous unmanned aerial vehicle (UAV) are described. In an example embodiment, a UAV can be configured to track a human subject using perception inputs from one or more onboard sensors. The perception inputs can be utilized to generate values for various performance metrics associated with the activity of the human subject. In some embodiments, the perception inputs can be utilized to autonomously maneuver the UAV to lead the human subject to satisfy a performance goal. The UAV can also be configured to autonomously capture images of a sporting event and/or make rule determinations while officiating a sporting event.

First claim

What is claimed is:

1 . A method for facilitating fitness training using an unmanned aerial vehicle (UAV), the method comprising:
receiving, by a computer system, a behavioral objective input, the behavioral objective input configured based on a performance goal for a human subject; receiving, by a computer system, perception inputs generated by one or more sensors associated with the UAV; generating, by the computer system, a planned trajectory through a physical environment based on the behavioral objective input and the perception inputs, the planned trajectory configured to cause the UAV to lead the human subject so as to satisfy the performance goal; and causing, by the computer system, the UAV to autonomously maneuver along the planned trajectory.

2 . The method of claim 1 , wherein the performance goal defines a particular running pace and wherein causing the UAV to maneuver along the planned trajectory includes causing the UAV to fly at a speed that sets the particular pace for the human subject.

3 . The method of claim 1 , wherein generating the planned trajectory includes:
processing, by computer system, the behavioral objective with one or more other behavioral objectives using any of gradient-based optimization, gradient-free optimization, sampling, or end-to-end.

4 . The method of claim 1 , further comprising:
determining, by the computer system, based on the perception inputs, that the human subject is injured and/or that the human subject is tired; and adjusting, by the computer system, the planned trajectory in response to determining that the human subject is injured and/or that the human subject is tired.

5 . The method of claim 1 , further comprising:
determining, by the computer system, based on the perception inputs, a terrain characteristic of the physical environment and adjusting the planned trajectory in response to determining the terrain characteristic of the physical environment.

6 . The method of claim 1 , wherein the behavioral objective input is based on tracking data generated by a second UAV tracking a second human subject, the tracking data indicative of the motion of the second human subject.

7 . The method of claim 6 , wherein the planned trajectory corresponds with the motion of the second human subject.

8 . The method of claim 1 , further comprising:
generating, by the computer system, based on the perception inputs, tracking data indicative of the motion of the human subject; and transmitting, by the computer system, via a wireless communication link, the tracking data to a second UAV; wherein the tracking data is utilized by the second UAV to autonomously maneuver along a second planned trajectory that corresponds with the motion of the human subject.

9 . The method of claim 1 , wherein the perception inputs include images captured by a camera coupled to the UAV.

10 . The method of claim 1 , wherein the performance goal is any of a particular pace, a particular speed, a particular time, or a particular distance.

11 . The method of claim 1 , wherein the human subject is any of a runner, a swimmer, a bicyclist, a skier, or a snowboarder.

12 . A method for facilitating fitness training using an unmanned aerial vehicle (UAV), the method comprising:
receiving, by a computer system, images of a physical environment captured by one or more image capture devices associated with the UAV, the UAV in autonomous flight through the physical environment in proximity to a human subject; processing, by the computer system, the received images to detect and track a motion of the human subject through the physical environment; analyzing, by the computer, system, the motion of the human subject based on the tracking; generating, by the computer system, based on the analysis, a value for a performance metric associated with the motion of the human subject; and causing display, by the computer system, of a visual output that includes at least some of the images of the physical environment and an indication of the value of the performance metric.

13 . The method of claim 12 , further comprising:
causing display, by the computer system, of the visual output at a mobile device.

14 . The method of claim 13 , wherein the mobile device is communicatively coupled to the UAV via a wireless communication link and wherein the visual output is displayed at the mobile device in real-time as the UAV is in autonomous flight and tracking the human subject.

15 . The method of claim 12 , further comprising:
generating, by the computer system, a graphical element based on the analysis of the motion of the human subject; wherein the visual output includes the graphical element.

16 . The method of claim 15 , wherein the graphical element includes any of:
a graphical representation of a trajectory of the human subject; or a graphical representation of a skeletal structure of the human subject.

17 . The method of claim 12 , wherein the performance metric includes any of speed, total run time, lap time, gait, pace, or elevation gain.

18 . The method of claim 12 , wherein the human subject is any of a runner, a swimmer, a bicyclist, a skier, or a snowboarder.

19 . A method for capturing images of a sporting event using an unmanned aerial vehicle (UAV), the method comprising:
receiving, by a computer system, perception inputs generated by one or more sensors associated with the UAV, the one or more sensors including an image capture device; detecting and tracking, by the computer system, based on the perception inputs, a moving area of interest within a particular area associated with the sporting event; and causing, by the computer systems, the UAV to autonomously maneuver and adjust an orientation of the image capture device to keep the area of interest in a field of view of the image capture device.

20 . The method of claim 19 , wherein the particular area associated with the sporting event is a field of play.

21 . The method of claim 19 , wherein the moving area of interest corresponds with the motion of a ball in play.

22 . The method of claim 19 , wherein causing the UAV to autonomously maneuver includes:
generating and continually updating, by the computer system, based on the perception inputs, a planned trajectory configured to keep the UAV within a threshold proximity of the moving area of interest while simultaneously avoiding overflying the particular area of the sporting event.

23 . The method of claim 19 , further comprising:
causing display, by the computer system, of images from the image capture device at a display device.

24 . The method of claim 19 , further comprising:
accessing, by the computer system, a rule associated with the sporting event; and processing, by the computer system, the perception inputs to detect an activity occurring during the sporting event; and applying, by the computer system, the accessed rule to the detected activity to generate a rule determination.

25 . The method of claim 24 , further comprising:
causing, by the computer system, the UAV to generate an audible output indicative of the rule determination; and/or causing, by the computer system, a public address system at a venue hosting the sporting event to generate an audible output indicative of the rule determination.

26 . The method of claim 24 , further comprising:
receiving, by the computers syste

Assignees

• Skydio Inc

Inventors

• Bachrach Abraham Galton
• Bry Adam Parker
• Donahoe Matthew Joseph
• Martirosyan Hayk
• Moss Tom

Classifications

• G06V20/17

  taken from planes or by drones · CPC title

• B64U2201/10

  autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS] · CPC title

• B64U2101/30

  for imaging, photography or videography · CPC title

• G06T7/20

  Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51) · CPC title

• G06T2207/30224

  Ball; Puck · CPC title