Providing customized entertainment experience using human presence detection

Patent metadata

Field	Value
Publication number	US-11202121-B2
Application number	US-202015930704-A
Country	US
Kind code	B2
Filing date	May 13, 2020
Priority date	May 13, 2020
Publication date	Dec 14, 2021
Grant date	Dec 14, 2021

Abstract

Disclosed herein are system, method, and computer program product embodiments for the detection of human presence in front of a plurality of sensors such as those of speakers and a device with a processor, such as a television. Data gathered from the plurality of sensors may be analyzed by the processor to determine if one or more humans are present proximate to the device. Based on the determined presence or absence of one or more humans, further actions including, inter alia, customizing a home theatre experience for the one or more humans, making content recommendations, or activating parental controls can be taken by the device.

First claim

What is claimed is:

1. A computer implemented method, comprising:
executing a collection routine at regular intervals to gather raw data from a plurality of sensors, wherein the executing the collection routine comprises varying a respective signal strength of one or more of the plurality of sensors;
receiving results of the collection routine in a form of the raw data from the plurality of sensors;
storing the results of the collection routine in a central data repository;
generating a three-dimensional composite map of signal strength based on the stored results;
analyzing, by at least one processor, the three-dimensional composite map of signal strength to determine if one or more humans are present proximate to the at least one processor within a predetermined geographical range;
in response to determining, based on the analyzing the three-dimensional composite map of signal strength, that the one or more humans are present,
determining, by the at least one processor and based on the three-dimensional composite map of signal strength, a respective geographic position of the one or more humans, and
determining, based on the stored results of the collection routine, if media content is being consumed by the one or more humans; and

executing at least one further action based on the geographical position of the one or more humans or based on the media content being consumed.

2. The method of claim 1 , wherein the plurality of sensors are in geographic proximity to the at least one processor.

3. The method of claim 1 , wherein the plurality of sensors form a high-resolution detection zone in front of the at least one processor, in 2 or 3 dimensions in a shape configurable by placement of the plurality of sensors.

4. The method of claim 1 , wherein the analyzing the three-dimensional composite map of signal strength to determine if the one or more humans are proximate to the at least one processor within the predetermined geographical range further comprises:
providing the raw data as input to a neural network machine learning classifier, the neural network machine learning classifier having an input layer that receives the raw data as a plurality of inputs, and an output layer; and
comparing values of nodes of the output layer to determine a presence or absence of the one or more humans proximate to the at least one processor in the predetermined geographical range.

5. The method of claim 1 , wherein the executing the at least one further action based on the geographical position of the one or more humans or based on the media content being consumed further comprises:
executing commands to generate a graphic user interface (GUI); and
displaying on the GUI a preferable speaker arrangement position for optimal sound quality based on the respective geographic position of the one or more humans.

6. The method of claim 1 , wherein the executing the at least one further action based on the geographical position of the one or more humans or based on the media content being consumed further comprises:
executing commands to control lighting with respect to brightness and angle relative to the one or more humans.

7. The method of claim 1 , wherein the executing the at least one further action based on the geographical position of the one or more humans or based on the media content being consumed further comprises:
executing commands to make content recommendations or activate parental controls based on the content.

8. A system, comprising:
a memory; and
at least one processor coupled to the memory and configured to:
execute a collection routine at regular intervals to gather raw data from a plurality of sensors, wherein to execute the collection routine, the at least one processor is configured to vary a respective signal strength of one or more of the plurality of sensors;
receive results of the collection routine in a form of the raw data from the plurality of sensors;
store the received results in a central data repository in the memory;
generate a three-dimensional composite map of signal strength based on the stored results;
analyze the three-dimensional composite map of signal strength to determine if one or more humans are present proximate to the at least one processor within a predetermined geographical range;
in response to a determination, based on an analysis of the three-dimensional composite map of signal strength, that the one or more humans are,
determine, based on the three-dimensional composite map of signal strength, a respective geographic position of the one or more humans, and
determine, based on the stored results of the collection routine, if media content is being consumed by the one or more humans; and

execute at least one further action based on the geographical position of the one or more humans or based on the media content being consumed.

9. The system of claim 8 , wherein the plurality of sensors are in geographic proximity to the at least one processor.

10. The system of claim 8 , wherein the plurality of sensors form a high-resolution detection zone in front of the at least one processor, in 2 or 3 dimensions in a shape configurable by placement of the plurality of sensors.

11. The system of claim 8 , wherein to analyze the three-dimensional composite map of signal strength to determine if the one or more humans are proximate to the at least one processor within the predetermined geographical range, the at least one processor is further configured to:
provide the raw data as input to a neural network machine learning classifier, the neural network machine learning classifier having an input layer that receives the raw data as a plurality of inputs, and an output layer; and
compare values of nodes in the output layer to determine a presence or absence of one or more humans proximate to the at least one processor in the predetermined geographical range.

12. The system of claim 8 , wherein to execute the at least one further action based on the geographical position of the one or more humans or based on the media content being consumed, the at least one processor is further configured to:
execute commands to generate a graphic user interface (GUI); and
display, on the GUI, a preferable speaker arrangement position for optimal sound quality based on the respective geographic position of the one or more humans.

13. The system of claim 8 , wherein to execute the at least one further action based on the geographical position of the one or more humans or based on the media content being consumed, the at least one processor is further configured to:
execute commands to control lighting with respect to brightness and angle relative to the one or more humans.

14. The system of claim 8 , wherein to execute the at least one further action based on the geographical position of the one or more humans or based on the media content being consumed, the at least one processor is further configured to:
analyze further for each of the one or more humans;
execute commands to make content recommendations or activate parental controls based on the content.

15. A non-transitory computer-readable medium having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:
executing a collection routine at regular intervals to gather raw data from a plurality of sensors, wherein the executing the collection routine comprises varying a respective signal strength of one or more of the plurality of sensors;
receiving results of the collection routine in a form of the raw data from the

Assignees

• Roku Inc

Inventors

• Neerbek Jan
• Malewski Rafal Krzysztof
• Møller Brian Thoft Moth
• Nangeroni Paul
• Chari Amalavoyal Narasimha

Classifications

• G06N3/048

  Activation functions · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0499

  Feedforward networks · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• H04N21/44218 Primary

  Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45) · CPC title