User aware automated content capture and augmentation

Patent metadata

Field	Value
Publication number	US-12568271-B2
Application number	US-202218560293-A
Country	US
Kind code	B2
Filing date	May 17, 2022
Priority date	May 19, 2021
Publication date	Mar 3, 2026
Grant date	Mar 3, 2026

Abstract

A method performed by a first electronic device includes receiving from a second electronic device an indication of an occurrence of one or more trigger conditions for content capture of multimedia content displayed on the first device, starting capture of the multimedia content in response to the trigger condition indication, collecting information from a third device, wherein the information from the third device is response information that is related to the multimedia content displayed on the first device, augmenting the captured multimedia content with the collected information from the third device for simultaneous display, and displaying the augmented multimedia content.

First claim

The invention claimed is:

1 . A method performed by a first electronic device, the method comprising:
receiving from a second electronic device an indication of an occurrence of one or more conditions for content capture of multimedia content displayed on the first electronic device; starting capture of the multimedia content in response to the indication; collecting response information from a third electronic device, wherein the response information is generated at the third electronic device as a reaction to the multimedia content displayed on the first electronic device; augmenting the captured multimedia content with the collected response information from the third device for simultaneous display; and providing for display of the augmented multimedia content.

2 . The method of claim 1 , wherein collecting response information from a third electronic device further comprises collecting information of a user's response to viewing the multimedia content displayed on the first electronic device.

3 . The method of claim 2 , wherein collecting information from the third device of response information comprises collecting any of notes, sensor readings, motions, video, or audio in response to the multimedia content displayed on the first electronic device.

4 . The method of claim 3 , wherein collecting information from the third device further comprises receiving timestamp information to align the response information with the captured multimedia content.

5 . The method of claim 1 , wherein receiving from a second electronic device an indication of an occurrence of one or more conditions comprises receiving any of: a user command, a user physical absence from a media consumption device (MCD), a user's status or activity, a user's input to the first or second electronic device, a user's predicted physical absence from the MCD, a user's environment, and a user's specified content occurrence.

6 . The method of claim 1 , wherein augmenting the captured multimedia content with the collected response information is performed according to an augmentation setting that defines a combination of the response information with the multimedia content.

7 . The method of claim 6 , wherein the combination of the response information with the multimedia content comprises an overlay of any of video and audio with the multimedia content.

8 . The method of claim 1 , wherein providing for display of the augmented multimedia content comprises providing a rendering on either the first electronic device or the third device for playback immediately or at a later time.

9 . The method of claim 1 , further comprising collecting feedback information from a user and updating the conditions based on the collected feedback.

10 . An electronic device, comprising at least one processor and circuitry configured to:
receive from a second electronic device an indication of an occurrence of one or more conditions for content capture of multimedia content displayed on the electronic device; start capture of the multimedia content in response to the indication; collect response information from a third electronic device, wherein the response information is generated at the third electronic device as a reaction to the multimedia content displayed on the electronic device; augment the captured multimedia content with the collected response information from the third device for simultaneous display; and provide for display of the augmented multimedia content.

11 . The electronic device of claim 10 , wherein the electronic device collects response information of a user's response to viewing the multimedia content displayed on the electronic device.

12 . The electronic device of claim 10 comprising one or more a digital television, head-mounted device, personal computer, a computer tablet device, and a smart phone.

13 . The electronic device of claim 10 , wherein the electronic device receives an indication of one or more conditions for content capture via any of a user command, a user physical absence from a media consumption device (MCD), a user's status or activity, a user's input to the electronic device or second electronic device, a user's predicted physical absence from the MCD, a user's environment, and a user's specified content occurrence.

14 . The electronic device of claim 10 , wherein the electronic device collects, from the third electronic device, response information comprising any of notes, sensor readings, motions, video, or audio in response to the multimedia content displayed on the electronic device.

15 . The electronic device of claim 14 , wherein the electronic device augments the captured multimedia content with the response information according to an augmentation setting that defines a combination of the response information with the multimedia content.

16 . The electronic device of claim 15 , wherein the augmentation setting comprises information of an overlay of any of video and audio with the multimedia content.

17 . The electronic device of claim 14 , wherein the electronic device further collects timestamp information to align the response information with the captured multimedia content.

18 . The electronic device of claim 10 , wherein the electronic device provides for display of the augmented multimedia content for rendering on either the electronic device or the third electronic device for playback in real-time or at a later time.

19 . The electronic device of claim 10 , wherein the electronic device is further configured to collect feedback information from a user and update the conditions based on the collected feedback.

20 . The electronic device of claim 10 , wherein the electronic device is configured such that the second or the third electronic device could be a sensor or sensing device, and the electronic device and the second electronic device are combined into a single device or the second electronic device and the third electronic device are combined into a single device.

Assignees

• Interdigital Madison Patent Holdings Sas

Inventors

• Liu Lu
• Seed Dale
• Dougherty Paul
• Kim Sanghoon

Classifications

• H04N21/8547

  involving timestamps for synchronizing content · CPC title

• H04N21/44218

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title

• H04N21/4312

  involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations · CPC title

• H04N21/4532

  involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18) · CPC title

• H04N21/43615

  Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805) · CPC title