Using gestures to control a media player

Patent metadata

Field	Value
Publication number	US-12411556-B2
Application number	US-202418889616-A
Country	US
Kind code	B2
Filing date	Sep 19, 2024
Priority date	Oct 25, 2022
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

In one aspect, an example method includes (i) receiving, by a computing system and from an input device associated with the computing system, a command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform; (ii) in response to receiving the command, monitoring, by the computing system and using a camera, a viewing environment of the media player to detect performance by a person of the customized gesture; and (iii) in response to detecting performance of the customized gesture: generating, by the computing system, a classification for use by the computing system for detecting the customized gesture, and storing, by the computing system, in memory, mapping data that correlates the detected customized gesture with the particular action.

First claim

What is claimed is:

1. A computing system configured for performing a set of acts comprising:
receiving, from an input device associated with the computing system, a command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform;
in response to receiving the command, monitoring, using a first camera, a viewing environment of the media player to detect performance by a person of the customized gesture, wherein the first camera is mounted at a first location within the viewing environment of the media player;
determining training data of the person within the viewing environment of the media player performing a particular gesture;
based on the determined training data and further based on pose data specifying a known orientation of the first camera, generating a classification for use by the computing system for detecting the particular gesture using a second camera, wherein the second camera is mounted at a second location different from the first location and exterior to the viewing environment of the media player;
monitoring an environment of the second camera to detect the particular gesture;
correlating the detected particular gesture with a particular corresponding action of the plurality of actions; and
controlling the media player to perform the correlated particular corresponding action.

2. The computing system of claim 1 , wherein the first camera comprises a night vision camera.

3. The computing system of claim 1 , wherein the second camera comprises a night vision camera.

4. The computing system of claim 1 , wherein correlating the detected particular gesture with the particular corresponding action comprises:
comparing the detected particular gesture with a library of known gestures;
determining that the detected particular gesture has at least a threshold degree of similarity to a specific gesture within the library of known gestures; and
in response to determining that the detected particular gesture has at least the threshold degree of similarity to the specific gesture within the library of known gestures, selecting the particular action based on mapping data that correlates each gesture of the library of known gestures to a respective one of the plurality of actions.

5. The computing system of claim 1 , wherein:
the customized gesture comprises a person picking up a phone; and
the correlated particular action comprises pausing media content being presented for display by the media player.

6. The computing system of claim 1 , wherein the input device associated with the computing system comprises a microphone.

7. The computing system of claim 6 , wherein the command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform comprises a voice command.

8. The computing system of claim 1 , wherein the input device associated with the computing system comprises a remote control associated with the media player.

9. The computing system of claim 1 , wherein detecting performance of the customized gesture by the person comprises:
based on data received from one or more sensors in the viewing environment, detecting the person within one or more images of the viewing environment captured by the camera of the one or more sensors;
in response to detecting the person:
loading, from memory, a gesture profile associated with the detected person, wherein the gesture profile comprises user-specified mapping data that correlates each gesture of a library of gestures to a respective one of the plurality of actions, and
monitoring, using the camera, the viewing environment to detect performance by the person of the customized gesture; and

correlating the detected customized gesture to a particular action of the plurality of actions that the mapping data of the gesture profile correlates to the detected customized gesture.

10. The computing system of claim 1 , wherein the computing system comprises a controller onboard the media player.

11. A method comprising:
receiving, from an input device associated with a computing system, a command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform;
in response to receiving the command, monitoring, using a first camera, a viewing environment of the media player to detect performance by a person of the customized gesture, wherein the first camera is mounted at a first location within the viewing environment of the media player;
determining training data of the person within the viewing environment of the media player performing a particular gesture;
based on the determined training data and further based on pose data specifying a known orientation of the first camera, generating a classification for use by the computing system for detecting the particular gesture using a second camera, wherein the second camera is mounted at a second location different from the first location and exterior to the viewing environment of the media player;
monitoring an environment of the second camera to detect the particular gesture;
correlating the detected particular gesture with a particular corresponding action of the plurality of actions; and
controlling the media player to perform the correlated particular corresponding action.

12. The method of claim 11 , wherein the first camera comprises a night vision camera.

13. The method of claim 11 , wherein the second camera comprises a night vision camera.

14. The method of claim 11 , wherein correlating the detected particular gesture with the particular corresponding action comprises:
comparing the detected particular gesture with a library of known gestures;
determining that the detected particular gesture has at least a threshold degree of similarity to a specific gesture within the library of known gestures; and
in response to determining that the detected particular gesture has at least the threshold degree of similarity to the specific gesture within the library of known gestures, selecting the particular action based on mapping data that correlates each gesture of the library of known gestures to a respective one of the plurality of actions.

15. The method of claim 11 , wherein:
the customized gesture comprises the person picking up a phone, and
the correlated particular action comprises pausing media content being presented for display by the media player.

16. The method of claim 11 , wherein the input device associated with the computing system comprises a microphone.

17. The method of claim 16 , wherein the command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform comprises a voice command.

18. The method of claim 11 , wherein the input device associated with the computing system comprises a remote control associated with the media player.

19. The method of claim 11 , wherein detecting performance of the customized gesture by the person comprises:
based on data received from one or more sensors in the viewing environment, detecting the person within one or more images of the viewing environment captured by the camera of the one or more sensors;
in response to detecting the person:
loading, from memory, a gesture profile associated with the detected person, wherein the gesture profile comprises user-specified mapping data that correlates each gesture of a library of gestures to a respective one of the plurality of actions, and
monitoring, using

Assignees

• Roku Inc

Inventors

• Cutter Michael
• Ramesh Sunil
• Levitian Karina

Classifications

• H04N23/90 Primary

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title