Method and system for generating a visual composition of user reactions in a shared content viewing session

Patent metadata

Field	Value
Publication number	US-12445682-B2
Application number	US-202418434135-A
Country	US
Kind code	B2
Filing date	Feb 6, 2024
Priority date	Jan 24, 2023
Publication date	Oct 14, 2025
Grant date	Oct 14, 2025

Abstract

In one aspect, an example method in connection with a shared content viewing session in which multiple users are receiving and viewing respective instances of the same media content in a synchronized manner is disclosed. The example method includes (i) detecting that each of the multiple users respectively exhibits a threshold extent of physical reaction around the same time; (ii) responsive to the detecting, for each of the multiple users, generating and/or storing respective visual content representing that user's physical reaction; (iii) generating a visual content composition that includes at least the generated and/or stored visual content for each of the multiple users; and (iv) outputting for presentation, the generated visual content composition.

First claim

The invention claimed is:

1. A method for use in connection with a shared content viewing session in which multiple users are receiving and viewing respective instances of media content in a synchronized manner, the method comprising:
accessing metadata associated with the media content, wherein the metadata specifies (i) a time point in the media content at which a particular event occurs, and (ii) an expected type of physical reaction associated with the particular event;
near the time point in the media content at which a particular event occurs, detecting that (i) each of the multiple users respectively exhibits a threshold extent of physical reaction around a same viewing time within the shared content viewing session, and (ii) the expected type of physical reaction matches a type of the physical reaction of the multiple users;
responsive to the detecting, for each of the multiple users, generating and/or storing respective visual content representing that user's physical reaction;
generating a visual content composition that includes at least the generated and/or stored visual content for each of the multiple users; and
outputting for presentation, the generated visual content composition.

2. The method of claim 1 , wherein exhibiting the threshold extent of physical reaction comprises exhibiting a threshold change in facial expression around the same time.

3. The method of claim 1 , wherein exhibiting the threshold extent of physical reaction comprises exhibiting a threshold change in body language expression around the same time.

4. The method of claim 1 , wherein around the same viewing time comprises within a given time period range.

5. The method of claim 4 , wherein the given time period range is between 0.5 seconds and 3.0 seconds.

6. The method of claim 1 , wherein detecting that each of the multiple users respectively exhibits a threshold extent of physical reaction around the same viewing time comprises:
for each of the multiple users:
receiving visual data of the user captured by a camera,
detecting a set of physical features in the visual data, and
based on the detected set of physical features, determining a physical reaction exhibited by the user using a physical reaction model comprising a classifier configured to map each of a plurality of physical reactions to a corresponding set of physical features.

7. The method of claim 1 , wherein for each of the multiple users, generating and/or storing respective visual content representing that user's physical reaction comprises storing respective visual content captured by a camera of that user.

8. The method of claim 1 , wherein for each of the multiple users, generating and/or storing respective visual content representing that user's physical reaction comprises generating and/or storing respective visual content of an avatar generated for that user.

9. The method of claim 1 , wherein the generated visual content composition further includes a portion of the media content that corresponds to a time point or range at or during which the users' physical reactions occurred.

10. The method of claim 1 , wherein outputting for presentation, the generated visual content composition comprises, transmitting the generated visual content composition to multiple content-presentation devices, respectively associated with the multiple users.

11. The method of claim 10 , wherein at least one of the multiple content-presentation devices is a television.

12. A computing system configured for performing a set of acts in connection a shared content viewing session in which multiple users are receiving and viewing respective instances of media content in a synchronized manner, the set of acts comprising:
accessing metadata associated with the media content, wherein the metadata specifies (i) a time point in the media content at which a particular event occurs, and (ii) an expected type of physical reaction associated with the particular event;
near the time point in the media content at which a particular event occurs, detecting that (i) each of the multiple users respectively exhibits a threshold extent of physical reaction around a same viewing time within the shared content viewing session, and (ii) the expected type of physical reaction matches a type of the physical reaction of the multiple users;
responsive to the detecting, for each of the multiple users, generating and/or storing respective visual content representing that user's physical reaction;
generating a visual content composition that includes at least the generated and/or stored visual content for each of the multiple users; and
outputting for presentation, the generated visual content composition.

13. The computing system of claim 12 , wherein exhibiting the threshold extent of physical reaction comprises exhibiting a threshold change in facial expression around the same time.

14. The computing system of claim 12 , wherein exhibiting the threshold extent of physical reaction comprises exhibiting a threshold change in body language expression around the same time.

15. The computing system of claim 12 , wherein around the same viewing time comprises within a given time period range.

16. The computing system of claim 15 , wherein the given time period range is between 0.5 seconds and 3.0 seconds.

17. The computing system of claim 12 , wherein detecting that each of the multiple users respectively exhibits a threshold extent of physical reaction around the same viewing time comprises:
for each of the multiple users:
receiving visual data of the user captured by a camera,
detecting a set of physical features in the visual data, and
based on the detected set of physical features, determining a physical reaction exhibited by the user using a physical reaction model comprising a classifier configured to map each of a plurality of physical reactions to a corresponding set of physical features.

18. The computing system of claim 12 , wherein for each of the multiple users, generating and/or storing respective visual content representing that user's physical reaction comprises storing respective visual content captured by a camera of that user.

19. The computing system of claim 12 , wherein for each of the multiple users, generating and/or storing respective visual content representing that user's physical reaction comprises generating and/or storing respective visual content of an avatar generated for that user.

20. A non-transitory computer-readable medium having stored thereon program instructions that upon execution by a computing system, cause performance of a set of acts in connection with a shared content viewing session in which multiple users are receiving and viewing respective instances of media content in a synchronized manner, the set of acts comprising:
accessing metadata associated with the media content, wherein the metadata specifies (i) a time point in the media content at which a particular event occurs, and (ii) an expected type of physical reaction associated with the particular event;
near the time point in the media content at which a particular event occurs, detecting that (i) each of the multiple users respectively exhibits a threshold extent of physical reaction around a same viewing time within the shared content viewing session, and (ii) the expected type of physical reaction matches a type of the physical reaction of the multiple users;
responsive to the detecting, for each of the multiple users, generating and/or storing respective visual content representing that us

Assignees

• Roku Inc

Inventors

• Jethwa Ronica
• Ramesh Sunil
• Cutter Michael
• Levitian Karina

Classifications

• G06V40/23

  Recognition of whole body movements, e.g. for sport training · CPC title

• H04N21/854

  Content authoring · CPC title

• G06V40/176

  Dynamic expression · CPC title

• H04N21/8146

  involving graphical data, e.g. 3D object, 2D graphics · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title