Smoothing video panning

Patent metadata

Field	Value
Publication number	US-10154228-B1
Application number	US-201514757419-A
Country	US
Kind code	B1
Filing date	Dec 23, 2015
Priority date	Dec 23, 2015
Publication date	Dec 11, 2018
Grant date	Dec 11, 2018

Abstract

Devices, systems and methods are disclosed for smoothing rendering data (e.g., panning data and/or magnification data) prior to rendering a video clip. The rendering data may be recorded using a first sampling rate and the system may determine Bézier points using a second sampling rate. The Bézier points may be aligned with the rendering data and may be used to generate smoothed rendering data. The smoothed rendering data may be used to render a video clip that may be displayed to a user. If the user inputs additional rendering data during playback of the video clip, the system may generate second smoothed rendering data. For example, the system may record additional rendering data for a second half of the video clip and generate second smoothed rendering data, including a first half of the first smoothed rendering data, based on the additional rendering data.

First claim

What is claimed is:

1. A computer-implemented method for generating a video clip, the method comprising:
receiving, from a remote device, panoramic video data comprising panoramic video frames that have a first aspect ratio greater than 2:1, the panoramic video data representing a wide field of view surrounding a panoramic image capture device;
displaying, during operation of a video editing application, the panoramic video data;
receiving, during operation of the video editing application, touch inputs that correspond to the panoramic video data;
generating raw panning data from the touch inputs using a first sampling rate, wherein the raw panning data includes a first data point indicating a first angle of view that corresponds to a first video frame of the panoramic video frames;
generating control points from the raw panning data using a second sampling rate smaller than the first sampling rate, wherein the control points include a first control point that:
corresponds to the first video frame, and
has a magnitude equal to a magnitude of the first data point;

determining a tangent of a portion of the raw panning data that corresponds to the first video frame;
generating panning curve data that corresponds to the control points and includes a first curve data point corresponding to the first video frame and indicating the first angle of view, wherein a slope of the panning curve data that corresponds to the first video frame is equal to the tangent;
sending, to the remote device, the panning curve data; and
receiving, from the remote device, a video clip that has a second aspect ratio smaller than the first aspect ratio and corresponds to portions of the panoramic video data indicated by the panning curve data.

2. The computer-implemented method of claim 1 , further comprising:
displaying, during operation of a video editing application, the video clip;
receiving, during operation of the video editing application, second touch inputs that correspond to the video clip;
generating second raw panning data from the second touch inputs using the first sampling rate, wherein the second raw panning data includes a second data point indicating a second angle of view that corresponds to the first video frame;
determining a second video frame corresponding to a beginning of the second raw panning data;
determining a first portion of the panning curve data ending at the second video frame;
generating second control points from the second raw panning data using the second sampling rate, wherein the second control points include a second control point that:
corresponds to the second video frame, and
has a magnitude equal to a magnitude of the second data point; and

generating second panning curve data that corresponds to the second control points and includes a second curve data point corresponding to the first video frame and indicating the second angle of view.

3. The computer-implemented method of claim 1 , further comprising:
receiving annotation data associated with the panoramic video data, the annotation data including first annotation data indicating characteristics associated with the first video frame;
selecting, from the first annotation data, a first face represented in the first video frame;
determining, from the first annotation data, first pixel coordinates corresponding to a position of the first face represented in the first video frame;
determining that the first data point of the raw panning data corresponds to the first pixel coordinates;
determining that the raw panning data corresponds to the first face;
determining, from the annotation data, position data corresponding to a position of the first face represented in the panoramic video data; and
generating the panning curve data using the control points and the position data.

4. The computer-implemented method of claim 1 , further comprising:
receiving raw magnification data generated using the first sampling rate that includes a second data point indicating a first magnification value corresponding to the first video frame;
generating second control points from the raw magnification data using the second sampling rate, wherein the second control points include a second control point that:
corresponds to the first video frame, and
has a magnitude equal to a magnitude of the second data point;

generating magnification curve data that corresponds to the second control points and includes a second curve data point that corresponds to the first video frame and indicates the first magnification value;
sending, to the remote device, the panning curve data and the magnification curve data; and
receiving, from the remote device, the video clip that has the second aspect ratio and corresponds to portions of the panoramic video data indicated by the panning curve data and the magnification curve data.

5. A computer-implemented method, comprising:
receiving input data that is associated with video data having a first aspect ratio greater than 2:1;
determining a first value from the input data that is associated with a first video frame of the video data, the first value indicating at least one of an angle of view or a magnification value associated with the first video frame;
generating, using the input data, a first control point that:
corresponds to the first video frame, and
has a magnitude equal to the first value;

determining a tangent of a portion of the input data associated with the first video frame; and
generating, using the first control point and the tangent, output curve data that indicates the first value associated with the first video frame, wherein the output curve data is generated by modifying the input data based on the tangent.

6. The computer-implemented method of claim 5 , further comprising:
sending, to a remote device, the output curve data; and
receiving, from the remote device, a video clip that:
has a second aspect ratio smaller than the first aspect ratio, and
corresponds to portions of the video data indicated by the output curve data.

7. The computer-implemented method of claim 5 , further comprising:
generating, using the output curve data, preview video data that:
has a second aspect ratio smaller than the first aspect ratio, and
corresponds to portions of the video data indicated by the output curve data;

displaying the preview video data;
receiving second input data that:
is associated with the preview video data, and
indicates a second value associated with the first video frame;

determining a second video frame associated with a beginning of the second input data;
determining a first portion of the output curve data ending at the second video frame;
generating, using the second input data, a second control point that:
corresponds to the second video frame, and
has a magnitude equal to the second value;

aligning the second control point with a tangent of the second input data associated with the second video frame; and
generating, using the first portion and the second control point, second output curve data that indicates the second value associated with the first video frame.

8. The computer-implemented method of claim 5 , further comprising:
receiving annotation data associated with the video data;
selecting, using the annotation data, a first object represented in the video data;
determining, using the annotation data, first pixel coordinates associated with the first object represented in the first video frame;
determining a second value corresponding to the first pixel coordinates;
determining a difference between the first value and the second value;
determining that the difference is below a threshold;
determining that the input data corresponds to the f

Assignees

• Amazon Tech Inc

Inventors

• Van Tuyl Bentley Peter
• Pearson Mark Eugene
• Hanson Robert James

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title

• G06V20/46 Primary

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G06K9/00744

  Physics · mapped topic

• H04N5/772 Primary

  the recording apparatus and the television camera being placed in the same enclosure · CPC title