Spherical video editing

Patent metadata

Field	Value
Publication number	US-10217488-B1
Application number	US-201715844089-A
Country	US
Kind code	B1
Filing date	Dec 15, 2017
Priority date	Dec 15, 2017
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

Systems and methods provide for editing of spherical video data. In one example, a computing device can receive a spherical video (or a video associated with an angular field of view greater than an angular field of view associated with a display screen of the computing device), such as by a built-in spherical video capturing system or acquiring the video data from another device. The computing device can display the spherical video data. While the spherical video data is displayed, the computing device can track the movement of an object (e.g., the computing device, a user, a real or virtual object represented in the spherical video data, etc.) to change the position of the viewport into the spherical video. The computing device can generate a new video from the new positions of the viewport.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
receiving, using one or more processors of a computing device, first video data including spherical video data;
displaying a portion of a frame of the first video data on a display screen associated with the computing device, wherein the portion of the frame represents a viewport into the first video data and the viewport is associated with an original position for each frame of the first video data;
determining, within a duration between when the computing device detects a first input associated with generating second video data from the first video data and a second input associated with stopping generation of the second video data, movement data associated with an object within a time period when the display screen displays the portion of the frame of the first video data;
determining a plurality of new positions for the viewport using the movement data, wherein a new position corresponds to a movement associated with the object within the time period when the display screen displays the portion of the frame of the first video data; and
generating the second video data from the first video data using the plurality of new positions, wherein pixels of a frame of the second video data correspond to pixels of the frame of the first video data encompassed by the viewport at the new position.

2. The computer-implemented method of claim 1 , wherein the object includes the computing device.

3. The computer-implemented method of claim 2 , further comprising:
capturing at least one of motion sensor data or position sensor data of the computing device,
wherein the movement data is determined using at least one of the motion sensor data or the position sensor data.

4. The computer-implemented method of claim 3 , further comprising:
capturing image data representing a physical environment of the computing device,
wherein the movement data is determined using a combination of the image data and at least one of the motion sensor data or the position sensor data.

5. The computer-implemented method of claim 1 , wherein the object is displayed in the first video data.

6. The computer-implemented method of claim 1 , wherein the plurality of new positions corresponds to a plurality of centroids of the viewport, and wherein generating the second video data from the first video data includes re-centering the viewport using the plurality of centroids.

7. The computer-implemented method of claim 1 , further comprising:
determining at least one of rotation information, translation information, or transformation information for re-centering the viewport from the original position to the new position.

8. The computer-implemented method of claim 1 , further comprising:
determining a surface or a volume to extract from the frame of the first video data for the frame of the second video data.

9. The computer-implemented method of claim 1 , wherein the second video data includes an equirectangular frame having an angular field of view corresponding to an angular field of view of the display screen.

10. The computer-implemented method of claim 1 , wherein the frame of the second video data is associated with an angular field of view along at least one dimension that is greater than an angular field of view along at least one corresponding dimension of a display screen of the computing device but less than 360 degrees.

11. The computer-implemented method of claim 10 , wherein the frame of the second video data is associated with an angular field of view equal to 360 degrees along at least one second dimension.

12. The computer-implemented method of claim 1 , further comprising:
determining available resources of a second computing device; and
transmitting, based on the available resources, at least one of a cropped version of the second video data, a version of the second video data having a lower video resolution, a version of the second video data have a lower rate of frames per second, or a version of the second video data having regions of varying video resolutions.

13. A computing device, comprising:
one or more processors;
a display screen; and
memory including instructions that, upon execution by the one or more processors, cause the computing device to:
receive first video data associated with an angular field of view greater than an angular field of view associated with the display screen;
display a portion of a frame of the first video data on the display screen, wherein the portion of the frame represents a viewport into the first video data and the viewport is associated with an original position for each frame of the first video data;
receive a first input to the computing device associated with generating second video data from the first video data;
determine movement data associated with an object within a time period when the display screen displays the portion of the frame of the first video data;
determine a plurality of new positions for the viewport using the movement data, wherein a new position corresponds to a movement associated with the object within the time period when the display screen displays the portion of the frame of the first video data; and
generate the second video data from the first video data using the plurality of new positions, wherein pixels of a frame of the second video data correspond to pixels of the frame of the first video data encompassed by the viewport at the new position.

14. The computing device of claim 13 , further comprising a plurality of cameras, wherein the instructions upon execution further cause the computing device to:
capture a plurality of videos using the plurality of cameras; and
stitch the plurality of videos together to generate the first video data.

15. The computing device of claim 13 , wherein the object includes at least a portion of a user of the computing device.

16. The computing device of claim 15 , wherein the instructions upon execution further cause the computing device to:
determine a plurality of coordinates for mapping the pixels of the frame of the first video data to the pixels of the frame of the second video data.

17. A non-transitory computer-readable storage medium storing instructions that, upon execution by one or more processors of a computing device, cause the computing device to:
receive first video data including spherical video data;
receive a first input to the computing device associated with generating second video data from the first video data;
determine movement data associated with an object within a time period when the computing device displays a portion of a frame of the first video data, wherein the portion of the frame represents a viewport into the first video data and the viewport is associated with an original position for each frame of the first video data;
determine a plurality of new positions for the viewport using the movement data, wherein a new position corresponds to a movement associated with the object within the time period when the computing device displays the portion of the frame of the first video data; and
generate the second video data from the first video data using the plurality of new positions, wherein pixels of a frame of the second video data correspond to pixels of the frame of the first video data encompassed by the viewport at the new position.

18. The non-transitory computer-readable storage medium of claim 17 , wherein the instructions upon execution further cause the computing device to:
retrieve the pixels of the frame of the first video dat

Assignees

• Snap Inc

Inventors

• Huang Justin

Classifications

• H04N9/8042

  involving data reduction · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• H04N5/772

  the recording apparatus and the television camera being placed in the same enclosure · CPC title

• H04N9/8205

  involving the multiplexing of an additional signal and the colour video signal · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title