Spherical video editing

Patent metadata

Field	Value
Publication number	US-11380362-B2
Application number	US-202117319425-A
Country	US
Kind code	B2
Filing date	May 13, 2021
Priority date	Dec 15, 2017
Publication date	Jul 5, 2022
Grant date	Jul 5, 2022

Abstract

Systems and methods provide for editing of spherical video data. In one example, a computing device can receive a spherical video (or a video associated with an angular field of view greater than an angular field of view associated with a display screen of the computing device), such as by a built-in spherical video capturing system or acquiring the video data from another device. The computing device can display the spherical video data. While the spherical video data is displayed, the computing device can track the movement of an object (e.g., the computing device, a user, a real or virtual object represented in the spherical video data, etc.) to change the position of the viewport into the spherical video. The computing device can generate a new video from the new positions of the viewport.

First claim

What is claimed is:

1. A method comprising:
accessing, by one or more processors, a sequence of spherical frames;
detecting, by the one or more processors, a path of motion;
determining, by the one or more processors, portions of the spherical frames based on the path of motion, each portion being of a different spherical frame among the sequence of spherical frames based on a different position along the path of motion; and
generating, by the one or more processors, a video that includes the portions determined based on the path of motion.

2. The method of claim 1 , wherein:
the detecting of the path of motion includes capturing motion sensor data that indicates device movement along at least a portion of the path of motion.

3. The method of claim 1 , wherein:
the detecting of the path of motion includes capturing position sensor data that indicates device movement along at least a portion of the path of motion.

4. The method of claim 1 , wherein:
the detecting of the path of motion includes capturing optical sensor data that indicates device movement along at least a portion of the path of motion.

5. The method of claim 1 , wherein:
the detecting of the path of motion includes detecting the path of motion during activation of a control element that corresponds to maintenance of an editing mode of a device, the activation of the control element indicating that the path of motion is an editing input.

6. The method of claim 1 , wherein:
the detecting of the path of motion includes detecting the path of motion in response to activation of a control element that corresponds to entering an editing mode of a device, the activation of the control element indicating that the path of motion is an editing input.

7. The method of claim 1 , wherein:
the sequence of spherical frames are included in a spherical video; and
the detecting of the path of motion is in response to designation of the path of motion as an editing input.

8. The method of claim 1 , wherein:
the detecting of the path of motion includes tracking movement of an object depicted in the spherical video, the path of motion being defined by the tracked movement of the object depicted in the spherical video.

9. The method of claim 1 , wherein:
the detecting of the path of motion includes tracking movement of an object detected by a sensor of a device, the path of motion being defined by the tracked movement of the object detected by the sensor of the device.

10. The method of claim 1 , wherein:
the detecting of the path of motion includes tracking movement of an eye of a user of a device, the eye being detected by a sensor of the device, the path of motion being defined by the tracked movement of the eye of the user.

11. The method of claim 1 , wherein:
the detecting of the path of motion includes tracking movement of a head a user of a device, the head being detected by a sensor of the device, the path of motion being defined by the tracked movement of the head of the user.

12. A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising
accessing a sequence of spherical frames;
detecting a path of motion;
determining portions of the spherical frames based on the path of motion, each portion being of a different spherical frame among the sequence of spherical frames based on a different position along the path of motion; and
generating a video that includes the portions determined based on the path of motion.

13. The non-transitory machine-readable storage medium of claim 12 , wherein:
the detecting of the path of motion includes detecting the path of motion during activation of a control element that corresponds to maintenance of an editing mode of a device, the activation of the control element indicating that the path of motion is an editing input.

14. The non-transitory machine-readable storage medium of claim 12 , wherein:
the detecting of the path of motion includes detecting the path of motion in response to activation of a control element that corresponds to entering an editing mode of a device, the activation of the control element indicating that the path of motion is an editing input.

15. The non-transitory machine-readable storage medium of claim 12 , wherein:
the sequence of spherical frames are included in a spherical video; and
the detecting of the path of motion is in response to designation of the path of motion as an editing input.

16. The non-transitory machine-readable storage medium of claim 12 , wherein:
the detecting of the path of motion includes tracking movement of an object depicted in the spherical video, the path of motion being defined by the tracked movement of the object depicted in the spherical video.

17. A system comprising:
one or more processors; and
a memory storing instructions that, when executed by at least one processor among the one or more processors, cause the system to perform operations comprising:
accessing a sequence of spherical frames;
detecting a path of motion;
determining portions of the spherical frames based on the path of motion, each portion being of a different spherical frame among the sequence of spherical frames based on a different position along the path of motion; and
generating a video that includes the portions determined based on the path of motion.

18. The system of claim 17 , wherein:
the detecting of the path of motion includes detecting the path of motion during activation of a control element that corresponds to maintenance of an editing mode of a device, the activation of the control element indicating that the path of motion is an editing input.

19. The system of claim 17 , wherein:
the detecting of the path of motion includes detecting the path of motion in response to activation of a control element that corresponds to entering an editing mode of a device, the activation of the control element indicating that the path of motion is an editing input.

20. The system of claim 17 , wherein:
the sequence of spherical frames are included in a spherical video; and
the detecting of the path of motion is in response to designation of the path of motion as an editing input.

Assignees

• Snap Inc

Inventors

• Huang Justin

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N21/845

  Structuring of content, e.g. decomposing content into time segments · CPC title

• G06T7/73

  using feature-based methods · CPC title