Computerized system and method for in-video modification

Patent metadata

Field	Value
Publication number	US-11620825-B2
Application number	US-202217746074-A
Country	US
Kind code	B2
Filing date	May 17, 2022
Priority date	Apr 30, 2018
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

Disclosed are systems and methods for improving interactions with and between computers in content providing, searching and/or hosting systems supported by or configured with devices, servers and/or platforms. The disclosed systems and methods provide a novel framework for performing automatic detection of surfaces in video frames resulting in the creation of a seamless in-video ad experience for viewing users. The disclosed framework operates by leveraging available surfaces in videos to show advertisements in compliance with publisher protection, compliance and policy in a fully automatic, end-to-end solution. The disclosed framework evidences a streamlined, automatic and computationally efficient process(es) that modifies digital content at the surface level within the frames of the content in compliance with the digital rights of the owners of the content being merged via the disclosed augmentation.

First claim

What is claimed is:

1. A method comprising:
receiving, by a computing device, a video provided by a first user comprising content having a first set of constraints set by the first user defining write access to a portion of the content, the first constraints further comprising information indicating a maximum number of surfaces per frame that are writable;
analyzing, via the computing device, the video, and based on the analysis, identifying a first set of surfaces depicted within the content of the video;
comparing, via the computing device, the identified first set of surfaces and the first set of constraints, and based on the comparison, determining a manifest of surfaces, each surface identified within the manifest having attributes complying with the first set of constraints, the manifest being a number at or below the maximum number of surfaces;
identifying, via the computing device, a digital content item comprising attributes in compliance with a surface within the manifest;
modifying, via the computing device, the video using the identified digital content item; and
communicating, via the computing device, the modified video for rendering at a user computing device, the communication causing the modified video to be rendered such that the identified digital content item is displayed in association with the surface at the user computing device.

2. The method of claim 1 , the manifest of surfaces corresponding to the entire video.

3. The method of claim 1 , the manifest of surfaces corresponding to one or more individual scenes within the video.

4. The method of claim 1 , identifying a digital content item further comprising:
identifying, via the computing device, a subset of the manifest based on a second set of constraints, the subset comprising surfaces having attributes complying with definitions of the second set of constraints, wherein the surface is identified within the subset of manifest, the identified digital content item comprising attributes in compliance with the surface within the subset defined by the definitions of the second set of constraints.

5. The method of claim 4 , further comprising:
receiving, via the computing device, the second set of constraints from a second user, the second set of constraints comprising the definitions for adding second user provided digital content to surfaces of a video, the identified digital content item used to modify the video being digital content of the second user.

6. The method of claim 5 , the definitions of the second set of constraints defining a specific type of surface characteristic each surface must have for adding second user provided digital content.

7. The method of claim 5 , the definitions of the second set of constraints defining a specific type of scene a portion of video content must display for adding second user provided digital content.

8. The method of claim 5 , the definitions of the second set of constraints defining a specific location of a surface within a set of frames of the video content for adding second user provided digital content.

9. The method of claim 1 , the first set of constraints comprising definitions controlling types of content writable to the portion of the content and identifying which surfaces within the portion of the content are writable.

10. The method of claim 1 , modifying the video further comprising inserting the identified digital content within the content of the video in association with the surface.

11. The method of claim 1 , identifying the digital content item further comprising:
analyzing the video, and based on the analyzing, determining a context of the video;
searching a digital content database based on the context; and
identifying, based on the searching, the digital content item.

12. A non-transitory computer-readable storage medium tangibly encoded with computer-executable instructions that when executed by a processor associated with a computing device perform steps of a method comprising:
receiving a video provided by a first user comprising content having a first set of constraints set by the first user defining write access to a portion of the content, the first constraints further comprising information indicating a maximum number of surfaces per frame that are writable;
analyzing the video, and based on the analysis, identifying a first set of surfaces depicted within the content of the video;
comparing the identified first set of surfaces and the first set of constraints, and based on the comparison, determining a manifest of surfaces, each surface identified within the manifest having attributes complying with the first set of constraints, the manifest being a number at or below the maximum number of surfaces;
identifying a digital content item comprising attributes in compliance with a surface within the manifest;
modifying the video using the identified digital content item; and
communicating the modified video for rendering at a user computing device, the communication causing the modified video to be rendered such that the identified digital content item is displayed in association with the surface at the user computing device.

13. The non-transitory computer-readable storage medium of claim 12 , the manifest of surfaces corresponding to the entire video.

14. The non-transitory computer-readable storage medium of claim 12 , the manifest of surfaces corresponding to one or more individual scenes within the video.

15. The non-transitory computer-readable storage medium of claim 12 , identifying a digital content item further comprising:
identifying a subset of the manifest based on a second set of constraints, the subset comprising surfaces having attributes complying with definitions of the second set of constraints, wherein the surface is identified within the subset of manifest, the identified digital content item comprising attributes in compliance with the surface within the subset defined by the definitions of the second set of constraints.

16. The non-transitory computer-readable storage medium of claim 15 , the steps further comprising:
receiving the second set of constraints from a second user, the second set of constraints comprising the definitions for adding second user provided digital content to surfaces of a video, the identified digital content item used to modify the video being digital content of the second user.

17. The non-transitory computer-readable storage medium of claim 16 , the definitions of the second set of constraints defining a specific type of surface characteristic each surface must have for adding second user provided digital content, the definitions further defining a specific type of scene a portion of video content must display for adding second user provided digital content, and the definitions further defining a specific location of a surface within a set of frames of the video content for adding second user provided digital content.

18. The non-transitory computer-readable storage medium of claim 12 , the first set of constraints comprising definitions controlling types of content writable to the portion of the content and identifying which surfaces within the portion of the content are writable.

19. The non-transitory computer-readable storage medium of claim 12 , identifying the digital content item further comprising:
analyzing the video, and based on the analyzing, determining a context of the video;
searching a digital content database based on the context; and
identifying, based on the searching, the digital content item.

Assignees

• Yahoo Ad Tech Llc

Inventors

• Dassa Guy
• Zoller Whitney

Classifications

• G06T7/60

  Analysis of geometric attributes · CPC title

• G06V20/20 Primary

  in augmented reality scenes · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06Q30/0276

  Advertisement creation · CPC title

• H04N21/2668

  Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)} · CPC title