Method and apparatus for editing video scenes based on learned user preferences

Patent metadata

Field	Value
Publication number	US-9916862-B2
Application number	US-201615365842-A
Country	US
Kind code	B2
Filing date	Nov 30, 2016
Priority date	May 1, 2014
Publication date	Mar 13, 2018
Grant date	Mar 13, 2018

Abstract

A computer implemented method and apparatus for editing video scenes based on learned user preferences. The method comprises identifying a plurality of scenes in a video; for each scene of the plurality of scenes: applying an effect to a representative frame of a scene, wherein the effect is based on learned user preferences; receiving an input that adjusts at least one effect on the representative frame; updating user preferences based on received input; and applying the at least one effect to all frames of the scene; interpolating the adjusted at least one effect at a boundary of two consecutive scenes; and applying the interpolated at least one effect to transitional frames between the two consecutive scenes.

First claim

What is claimed is:

1. In a digital medium environment for generating edited digital videos, a computer implemented method comprising:
receiving an input that adjusts a first level of a digital effect of a first representative frame of a first scene of a plurality of scenes in a digital video, wherein each scene comprises a plurality of digital frames;
determining a user preference based on the received input;
modifying the plurality of digital frames of the first scene to have the adjusted level of the digital affect;
comparing a second representative frame of a second scene of the plurality of scenes in the digital video to the first representative frame;
determining a second level of the digital effect for the second scene based on the user preference and the comparison of the second representative frame of the second scene of the plurality of scenes in the digital video to the first representative frame, wherein the second level of the digital effect is different than the first level of the digital effect; and
modifying the plurality of digital frames of the second scene to have the determined second level of the digital effect.

2. The method of claim 1 , further comprising providing a user interface displaying a plurality of versions of the first representative frame and a pointer at a first position relative to the plurality of versions of the first representative frame, wherein:
the plurality of versions of the first representative frame comprise versions of the first representative frame having different levels of the digital effect; and
receiving the input that adjusts the level of the digital effect of the first representative frame of the first scene comprises receiving input modifying the first position of the pointer relative to the plurality of versions of the first representative frame to a second position relative to the plurality of versions of the first representative frame.

3. The method of claim 2 , wherein providing the user interface displaying the plurality of versions of the first representative frame comprises:
determining weights corresponding to a plurality of sample images based on a similarity of each sample image of the plurality of sample images to the first representative frame, wherein each sample image has corresponding adjustment parameters comprising an adjustment to the level of the digital effect;
determining a weighted average of the adjustment parameters based on the weights;
applying the weighted average of the adjustment parameters to the first representative frame to generate the plurality of versions of the first representative frame; and
providing the plurality of versions of the first representative frame to the user interface.

4. The method of claim 2 , wherein determining the user preference further comprises:
generating a first feature descriptor for the first representative frame; and
storing the first feature descriptor and the second position of the pointer as the user preference.

5. The method of claim 4 , wherein comparing the second representative frame of the second scene of the plurality of scenes in the digital video to the first representative frame comprises:
comparing the first feature descriptor for the first representative frame from the user preference to a second feature descriptor for the second representative frame to determine a similarity between the first representative frame and the second representative frame; and
generating a weight vector based on the similarity.

6. The method of claim 5 , wherein determining the second level of the digital effect for the second scene based on the comparison and the user preference comprises:
applying the weight vector to the second position of the pointer from the user preference corresponding to the first representative frame to determine an initial position of the pointer corresponding to the second representative frame; and
providing the user interface displaying a plurality of versions of the second representative frame and the pointer at the initial position.

7. The method of claim 6 , wherein determining the second level of the digital effect for the second scene based on the comparison and the user preference comprises:
receiving input modifying the initial position to a final position, wherein the final position of the pointer reflects the second level of the digital effect for the second scene.

8. The method of claim 7 , further comprising,
generating a modified feature descriptor for the second representative frame based on the second level of the digital effect; and
updating the user preference by storing the modified feature descriptor and the final position of the pointer with the first feature descriptor and the second position of the pointer.

9. The method of claim 1 , wherein the digital effect comprises two digital effects and each of the two digital effects comprise at least one of brightness, contrast, sharpness, or saturation.

10. A system comprising:
at least one processor; and
at least one non-transitory computer readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
receive an input that adjusts a level of a first digital effect of a first representative frame of a first scene of a plurality of scenes in a digital video, wherein each scene comprises a plurality of digital frames;
determine a user preference based on the received input;
modify the plurality of digital frames of the first scene to have the adjusted level of the digital affect;
compare a second representative frame of a second scene of the plurality of scenes in the digital video to the first representative frame;
determine a second level of the digital effect for the second scene based on the user preference and the comparison of the second representative frame of the second scene of the plurality of scenes in the digital video to the first representative frame, wherein the second level of the digital effect is different than the first level of the digital effect; and
modify the plurality of digital frames of the second scene to have the determined second level of the digital effect.

11. The system of claim 10 , further comprising instructions that, when executed by the at least one processor, cause the system to:
provide a user interface displaying a plurality of versions of the first representative frame and a pointer at a first position relative to the plurality of versions of the first representative frame, wherein the plurality of versions of the first representative frame comprise versions of the first representative frame having different levels of the digital effect; and
receive the input that adjusts the level of the digital effect of the first representative frame of the first scene by receiving input modifying the first position of the pointer relative to the plurality of versions of the first representative frame to a second position relative to the plurality of versions of the first representative frame.

12. The system of claim 11 , further comprising instructions that, when executed by the at least one processor, cause the system to determine the user preference by:
generating a first feature descriptor for the first representative frame; and
storing the first feature descriptor and the second position of the pointer as the user preference.

13. The system of claim 12 , further comprising instructions that, when executed by the at least one processor, cause the system to compare the second representative frame of the second scene of the plurality of scenes in the digital video to the first representative frame by:
comparing the first feature de

Assignees

• Adobe Systems Inc

Inventors

• Singhal Puneet

Classifications

• G06V10/7788

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title

• G06F18/41

  Interactive pattern learning with a human teacher · CPC title

• G06V20/48

  Matching video sequences · CPC title

• G06V20/46

  Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames · CPC title

• G11B27/031 Primary

  Electronic editing of digitised analogue information signals, e.g. audio or video signals · CPC title