Systems And Methods For Switching Of Video Streams Provided To A Ground Station From Various Sources Onboard An Unmanned Aerial Vehicle

Patent metadata

Field	Value
Publication number	US-2024179275-A1
Application number	US-202318522754-A
Country	US
Kind code	A1
Filing date	Nov 29, 2023
Priority date	Nov 29, 2022
Publication date	May 30, 2024
Grant date	—

Abstract

Systems, methods, and software for operating an unmanned aerial vehicle (UAV) having one or more camera sensors. A method includes capturing a video stream using the camera sensor(s) of the UAV. The method includes determining a capability of a ground station communicably coupled to the UAV to annotate the video stream. In response to determining that the ground station is capable to annotate the video stream, the method includes transmitting the video stream to the ground station without annotations. Alternatively, in response to determining that the ground station is incapable to annotate the video stream, drawing annotations on the video stream to generate an annotated video stream, and transmitting the annotated video stream to the ground station. Embodiments of the present technology provide enhanced interoperability between UAVs and different types of ground stations to improve streamlining and quality of user experiences as compared to conventional systems and techniques.

First claim

What is claimed is:

1 . A method of operating an unmanned aerial vehicle (UAV) having one or more camera sensors, the method comprising:
capturing a video stream using the one or more camera sensors of the UAV; determining a capability of a ground station communicably coupled to the UAV to annotate the video stream; and in response to determining that the ground station is capable to annotate the video stream, transmitting the video stream to the ground station without annotations; in response to determining that the ground station is incapable to annotate the video stream, drawing annotations on the video stream to generate an annotated video stream; and
transmitting the annotated video stream to the ground station.

2 . The method of claim 1 , further comprising:
establishing a communications link between the UAV and the ground station; and receiving, via the communications link, data representative of a capability of the ground station to annotate the video stream, wherein determining the capability of the ground station to annotate the video stream includes determining the capability of the ground station according to the data representative of a capability of the ground station to annotate the video stream.

3 . The method of claim 1 , further comprising:
encoding the video stream further in responsive to determining that the ground station is capable to annotate the video stream, wherein transmitting the video stream to the ground station without annotations includes transmitting the encoded video stream to the ground station.

4 . The method of claim 1 , further comprising:
generating data representative of one or more annotations for the video stream; and transmitting the data representative of one or more annotations to the ground station further in response to determining that the ground station is capable to annotate the video stream; wherein drawing annotations on the video stream includes drawing the annotations on an unencoded video stream according to the data representative of one or more annotations.

5 . The method of claim 4 , further comprising:
encoding the annotated video stream further in response to determining that the ground station is incapable to annotate the video stream, wherein transmitting the annotated video stream to the ground station comprises transmitting an encoded annotated video stream to the ground station.

6 . The method of claim 5 , further comprising:
receiving a first signal encoding data representative of a first toggling request from the ground station; and in response to receiving the first signal, first switching from transmitting the encoded annotated video stream to the ground station to transmitting the video stream to the ground station without annotations.

7 . The method of claim 6 , further comprising:
causing an encoder to produce an I-frame in the video stream without annotations off schedule further in response to receiving the first signal.

8 . The method of claim 6 , further comprising:
further in response to receiving the first signal,
monitoring a frame structure of the video stream without annotations; and
determining, based on the monitoring, a presence of an I-frame in the video stream without annotations.

9 . The method of claim 8 , wherein the first switching comprises switching from transmitting the encoded annotated video stream to the ground station to transmitting the video stream to the ground station without annotations upon determining the presence of the I-frame in the video stream without annotations.

10 . The method of claim 6 , further comprising:
receiving a second signal encoding data representative of a second toggling request from the ground station; and in response to receiving the second signal, second switching from transmitting the video stream without annotations to the ground station to transmitting the encoded annotated video stream to the ground station.

11 . The method of claim 10 , further comprising:
causing an encoder to produce an I-frame in the encoded annotated video stream off schedule further in response to receiving the second signal.

12 . The method of claim 10 , wherein the second switching comprises:
further in response to receiving the second signal:
monitoring a frame structure of the encoded annotated video stream; and
determining, based on the monitoring, a presence of an I-frame in the encoded annotated video stream,
wherein the second switching comprises switching from transmitting the video stream without annotations to the ground station to transmitting the encoded annotated video stream to the ground station upon determining the presence of the I-frame in the encoded annotated video stream.

13 . The method of claim 1 , wherein the one or more camera sensors include a plurality of navigational cameras, the method further comprising:
stitching together unencoded video streams from the plurality of navigational cameras to produce an unencoded stitched video stream; encoding the unencoded stitched video stream resulting in an encoded stitched video stream; and transmitting the encoded stitched video stream to the ground station.

14 . The method of claim 13 , wherein:
transmitting the encoded stitched video stream comprises transmitting the encoded stitched video stream to the ground station without annotations further in response to determining that the ground station is capable to annotate the video stream; or further in response to determining that the ground station is incapable to annotate the video stream, the method further comprises:
drawing annotations on the unencoded stitched video stream to generate an annotated unencoded stitched video stream,
wherein encoding the unencoded stitched video stream comprises encoding the annotated unencoded stitched video stream,
wherein transmitting the encoded stitched video stream comprises transmitting an encoded annotated unencoded stitched video stream to the ground station.

15 . The method of claim 14 , wherein the one or more camera sensors further include at least one imaging camera, the method further comprising:
receiving a third signal encoding data representative of a third toggling request from the ground station; and in response to receiving the third signal, and further in response to determining that the ground station is capable to annotate the video stream, third switching from transmitting the encoded stitched video stream to the ground station without annotations to transmitting the video stream to the ground station without annotations; or
further in response to determining that the ground station is incapable to annotate the video stream, fourth switching from transmitting the encoded annotated unencoded stitched video stream to the ground station to transmitting the annotated video stream to the ground station.

16 . The method of claim 14 , further comprising:
further in response to receiving the third signal, and further in response to determining that the ground station is capable to annotate the video stream, causing an encoder to produce an I-frame in the video stream to the ground station without annotations off schedule; or
further in response to determining that the ground station is incapable to annotate the video stream, causing an encoder to produce an I-frame in the encoded annotated video stream off schedule.

17 . The method of claim 14 , wherein, further in response to receiving the third signal:
the third switching comprises:
monitoring a frame structure of the video stream without annot

Assignees

• Skydio Inc

Inventors

• Moskovchenko Stepan
• Feng Jerry

Classifications

• H04N21/23424

  involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement · CPC title

• H04N21/2187

  Live feed · CPC title

• H04N21/23418

  involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59) · CPC title

• H04N21/23614

  Multiplexing of additional data and video streams (arrangements for simultaneous broadcast of plural pieces of information H04H20/28) · CPC title

• B64U10/13

  Flying platforms · CPC title