Method and apparatus for editing video scenes based on learned user preferences

Patent metadata

Field	Value
Publication number	US-9557829-B2
Application number	US-201414266987-A
Country	US
Kind code	B2
Filing date	May 1, 2014
Priority date	May 1, 2014
Publication date	Jan 31, 2017
Grant date	Jan 31, 2017

Abstract

A computer implemented method and apparatus for editing video scenes based on learned user preferences. The method comprises identifying a plurality of scenes in a video; for each scene of the plurality of scenes: applying an effect to a representative frame of a scene, wherein the effect is based on learned user preferences; receiving an input that adjusts at least one effect on the representative frame; updating user preferences based on received input; and applying the at least one effect to all frames of the scene; interpolating the adjusted at least one effect at a boundary of two consecutive scenes; and applying the interpolated at least one effect to transitional frames between the two consecutive scenes.

First claim

The invention claimed is:

1. A computer implemented method comprising:
identifying, by at least one processor, a plurality of scenes in a digital video, wherein each scene comprises a plurality of digital frames;
applying, by the at least one processor, a level of a digital effect to a representative frame of first scene of the digital video, wherein the level of the digital effect is based on learned user preferences;
receiving an input that adjusts the level of the digital effect on the representative frame of the first scene;
updating, by the at least one processor, the user preferences based on the received input;
modifying, by the at least one processor, a plurality of digital frames of the first scene to have the adjusted level of the digital effect; and
modifying a plurality of transitional frames at a boundary of the first scene and a next consecutive scenes in the digital video such that levels of the digital effect with regard to the transitional frames interpolate between the adjusted level of the digital effect of the first scene and a level of the digital effect of the next consecutive scene.

2. The method of claim 1 , wherein applying the level of the digital effect to the representative frame of the first scene comprises:
identifying a preferred level of the digital effect based on a level of the digital effect applied to one or more previously edited scenes; and
providing a pointer in a user interface at a position determined based on the identified preferred level of the digital effect, wherein the position of the pointer determines the level of the digital effect applied to the representative frame of the first scene.

3. The method of claim 2 , further comprising:
applying weights to a plurality of sample images based on a similarity of each sample image of the plurality of sample images to the representative frame;
generating a plurality of versions of the representative image based on the sample images by: applying weighted averages of adjustment parameters corresponding to the weighted sample images to the representative frame, wherein each of the plurality of versions comprises a version of the representative frame with an adjustment of the level of the digital effect based on the weighted averages of the adjustment parameters of the sample images; and
providing the plurality of versions of the representative images via the user interface; and
wherein providing the pointer in the user interface comprises: applying a weight to a previous final pointer position corresponding to at least one previously processed representative frame based on a similarity of the at least one previously processed representative frame to the representative frame.

4. The method of claim 2 , wherein updating the user preferences further comprises:
generating a feature descriptor for the representative frame; and
storing the feature descriptor and a final pointer position as user preferences, wherein the final pointer position comprises x,y coordinates that identify the level of the digital effect applied to the representative frame.

5. The method of claim 2 , further comprising applying an additional adjustment of the level of the digital effect to the representative frame based on a final position of the pointer.

6. The method of claim 1 , wherein modifying the plurality of transitional frames comprises:
determining the plurality of transitional frames by identifying at least one of:
(i) a predefined number of frames at an end of the first scene and a predefined number of frames at a beginning of the next consecutive scene, where the next consecutive scene immediately follows the first scene in the digital video; or
(ii) a predetermined percentage of frames at the end of the first scene and a predetermined percentage of frames at the beginning of the next consecutive scene, wherein the predetermined percentages are based on a number of frames in the first scene or a number of frames in the next consecutive scene.

7. The method of claim 1 , wherein the digital effect comprises two digital effects and further comprising: generating a plurality of extreme digital images for each combination of a high level and low level first digital effect and a high level and low level second digital effect of the two digital effects, and wherein each of the two digital effects comprises at least one of brightness, contrast, sharpness, or saturation.

8. A system comprising:
at least one processor; and
at least one non-transitory computer readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
identify a plurality of scenes in a digital video, wherein each scene comprises a plurality of digital frames;
apply a level of a digital effect to a representative frame of a first scene, wherein the level of the digital effect is based on learned user preferences;
receive an input that adjusts the level of the digital effect on the representative frame of the first scene;
update the user preferences based on the received input;
modify a plurality of digital frames of the first scene to have the adjusted level of the digital effect; and
modify a plurality of transitional frames at a boundary of the first scene and a next consecutive scene in the digital video such that the levels of the digital effect with regard to the transitional frames interpolate between the adjusted level of the digital effect of the first scene and a level of the digital effect of the next consecutive scene.

9. The system of claim 8 , wherein the instructions when executed by the at least one processor, cause the system to apply the level of the digital effect to the representative frame of the first scene by performing steps comprising:
identifying a preferred level of the digital effect based on a level of the digital effect applied to one or more previously edited scenes;
providing a pointer in a user interface at a position determined based on the identified preferred level of the digital effect, wherein the position of the pointer determines the level of the digital effect applied to the representative frame of the first scene; and
applying an additional adjustment of the level of the digital effect to the representative frame based on a final position of the pointer.

10. The system of claim 9 ,
further comprising instructions that, when executed by the at least one processor, cause the system to:
apply weights to a plurality of sample images based on a similarity of each sample image of the plurality of sample images to the representative frame;
generate a plurality of versions of the representative image based on the sample images by: applying weighted averages of adjustment parameters corresponding to the weighted sample images to the representative frame, wherein each of the plurality of versions comprises a version of the representative frame with an adjustment of the level of the digital effect based on the weighted averages of the adjustment parameters of the sample images; and
providing the plurality of versions via the user interface; and
wherein providing the pointer in the user interface comprises: applying a weight to a previous final pointer position corresponding to at least one previously processed representative frames based on a similarity of the at least one previously processed representative frame to the representative frame.

11. The system of claim 9 , wherein the instructions, when executed by the at least one processor, cause the system to update the user preference further by performing steps comprising:
generating a feature descriptor for the representative frame; and
storing the feature descriptor and a final pointer po

Assignees

• Adobe Systems Inc

Inventors

• Singhal Puneet

Classifications

• G06V10/7788

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title

• G06F18/41

  Interactive pattern learning with a human teacher · CPC title

• G06V20/48

  Matching video sequences · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• H04N5/2224

  related to virtual studio applications · CPC title