Replacing content of a surface in video

Patent metadata

Field	Value
Publication number	US-10306286-B2
Application number	US-201615195808-A
Country	US
Kind code	B2
Filing date	Jun 28, 2016
Priority date	Jun 28, 2016
Publication date	May 28, 2019
Grant date	May 28, 2019

Abstract

Replacing content of a surface in video is described. In one or more embodiments, a surface captured in a scene of video content is replaced with replacement content. As part of doing so, an indication of the surface to be replaced is received. The indication may be received as user input to select the surface for replacement from a frame of video content or as a pattern that can be used to search for the surface in frames of the video content. Once the surface is identified, it is tracked throughout the video content using motion tracking techniques. The motion tracking identifies frames of the video content that include the identified surface as well as locations of surface in the identified frames. Based on the identified frames and locations, the replacement content is incorporated into the video content to replace the surface.

First claim

What is claimed is:

1. A method implemented by a server device of a web service to expose viewers of video content to secondary content without disrupting a viewing experience of the viewers, the method comprising:
receiving, by the server device, an indication of a surface appearing in at least one scene of the video content, the indication indicating that the surface is to be replaced with replacement content, and the indication comprising one of:
a user selection input to select the surface for replacement from a frame of the video content; or
a pattern the server device uses to search for the surface in frames of the video content;

tracking, by the server device, the surface throughout the frames of the video content using motion tracking, including identifying frames that include the surface and locations of the surface in identified frames;
incorporating, by the server device, the replacement content into the identified frames at identified locations based on the tracking and by the server device replacing pixels of the identified frames at the identified locations with different pixels of an image depicting the replacement content;
generating, by the server device, replaced-surface video content in accordance with the incorporating, the replaced-surface video content configured to expose the replacement content in place of the surface when played back; and
communicating, by the server device, the replaced-surface video content for distribution to a viewer device, the communicated replaced-surface video content having the surface replaced in the at least one clip with the replacement content.

2. A method as described in claim 1 , wherein the tracking includes identifying both the frames that include the surface and the frames that do not include the surface.

3. A method as described in claim 1 , further comprising stabilizing an appearance of the replacement content across at least one of the frames into which the replacement content is incorporated by warping at least one of pixels of the replacement content or pixels of the at least one frame.

4. A method as described in claim 1 , further comprising:
collecting information that describes a viewer that requested the video content; and
determining the replacement content to incorporate into the video content based on the collected information.

5. A method as described in claim 4 , wherein the determining includes determining a market segment for the viewer based on the collected information, the determined replacement content being associated with the determined market segment.

6. A method as described in claim 1 , wherein the replacement content comprises advertising content.

7. A method as described in claim 1 , wherein the tracking includes searching the frames of the video content to identify the pattern.

8. A method as described in claim 7 , further comprising incorporating the replacement content at locations where the pattern is identified in the frames.

9. A method as described in claim 1 , wherein the web service is a web-based surface replacement service.

10. A method as described in claim 1 , further comprising communicating the replaced-surface video content to a client device for playback by the client device.

11. A method as described in claim 10 , wherein the replaced-surface video content is configured for playback by the client device without the client device incorporating the replacement content into the identified frames.

12. A method as described in claim 1 , wherein the video content is received by the server device from a content distributor responsive to a viewer request for the video content.

13. A server device to implement a web-based surface replacement service using cloud-based resources for exposing viewers of video content to secondary content without disrupting a viewing experience of the viewers, the server device comprising:
one or more processors; and
computer-readable storage media having stored thereon instructions that are executable by the one or more processors to implement a content replacement module to cause the server device to perform operations comprising:
obtaining the video content from a content distributor;
obtaining replacement content from a replacement content source;
incorporating the replacement content into the video content using the cloud-based resources and by replacing a surface that appears in at least one clip of the video content with the replacement content to generate replaced-surface video content, the surface being replaced by identifying the frames of the video content that include the surface and a location of the surface in the identified frames and by replacing pixels of the identified frames at the identified location with different pixels of an image depicting the replacement content; and
communicating the replaced-surface video content for distribution to a viewer device, the communicated replaced-surface video content having the surface replaced in the at least one clip with the replacement content.

14. A system as described in claim 13 , wherein the operations further comprise determining the replacement content to incorporate into the video content based on demographic information of a viewer.

15. A system as described in claim 13 , wherein the content replacement module utilizes a script-based motion tracking technique to identify the frames and the location.

16. A system as described in claim 13 , wherein the operations further comprise stabilizing an appearance of the replacement content for playback using content warping that adjusts pixels of at least one of the replacement content or the frames into which the replacement content is incorporated.

17. A system as described in claim 16 , wherein the content replacement module utilizes script-based content warping to stabilize the appearance of the replacement content.

18. A method implemented by a computing device for reducing distractions to video content viewing that are due to incorporation of advertising content by cloud-based resources of a web service, the method comprising:
requesting, by the computing device, video content from a content distributor;
receiving, by the computing device, a version of the video content in which a surface appearing in at least one scene of the video content has been replaced with advertising content using the cloud-based resources, the surface having been replaced by the cloud-based resources replacing pixels of frames of the video content including the surface with different pixels of an image depicting the advertising content; and
playing back, by the computing device, the version of replaced-surface video content effective to expose a viewer to the advertising content in place of the surface, the exposed advertising content appearing as part of the at least one scene based on the replacement.

19. A method as described in claim 18 , further comprising communicating demographic data describing the viewer to the cloud-based resource of the web service, the advertising content used to replace the surface being determined based on the demographic data.

20. A method as described in claim 18 , wherein:
the requesting corresponds to a request to stream the video content;
the receiving receives the version of the replaced-surface video content via streaming; and
the surface is replaced with the advertising content during at least a portion of the receiving.

Assignees

• Adobe Inc

Inventors

• Gupta Niraj
• Hussain Syed Saad
• Sharma Saurabh
• Agrawal Sandesh Kumar
• Gurfinkel Morgan O
• Tabak Jonathan David
• Loeffler Jens Markus
• Duggal Ashish

Classifications

• H04N21/812

  involving advertisement data (advertising per se G06Q30/02) · CPC title

• H04N21/237

  Communication with additional data server · CPC title

• H04N21/25883

  being end-user demographical data, e.g. age, family status or address (arrangements for identifying locations of users in broadcast systems H04H60/52) · CPC title

• H04N21/2665

  Gathering content from different sources, e.g. Internet and satellite · CPC title

• H04N21/6125

  involving transmission via Internet (transmission by internet of broadcast information H04H60/82) · CPC title