Using gestures to control a media player

Patent metadata

Field	Value
Publication number	US-12124635-B2
Application number	US-202318510953-A
Country	US
Kind code	B2
Filing date	Nov 16, 2023
Priority date	Oct 25, 2022
Publication date	Oct 22, 2024
Grant date	Oct 22, 2024

Abstract

In one aspect, an example method includes (i) receiving, by a computing system and from an input device associated with the computing system, a command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform; (ii) in response to receiving the command, monitoring, by the computing system and using a camera, a viewing environment of the media player to detect performance by a person of the customized gesture; and (iii) in response to detecting performance of the customized gesture: generating, by the computing system, a classification for use by the computing system for detecting the customized gesture, and storing, by the computing system, in memory, mapping data that correlates the detected customized gesture with the particular action.

First claim

What is claimed is:

1. A computing system configured for performing a set of acts comprising:
receiving, from an input device associated with the computing system, a command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform;
in response to receiving the command, monitoring, using a camera, a viewing environment of the media player to detect performance by a person of the customized gesture;
in response to detecting performance of the customized gesture:
generating a classification for use by the computing system for detecting the customized gesture, and
storing, in memory, mapping data that correlates the detected customized gesture with the particular action;

detecting multiple persons within one or more images of the viewing environment of the media player;
based on data received from one or more sensors in the viewing environment of the media player, selecting, from the multiple detected persons, a particular person to monitor for the customized gesture;
detecting performance of the customized gesture by the particular person; and
in response to detecting performance of the customized gesture by the particular person, controlling the media player to perform the correlated particular action.

2. The computing system of claim 1 , wherein the camera is a night vision camera.

3. The computing system of claim 1 , wherein the one or more sensors comprises a further camera.

4. The computing system of claim 1 , wherein the set of acts further comprises correlating the detected customized gesture with the correlated particular action, wherein the correlating comprises:
comparing the detected customized gesture with a library of known gestures,
determining that the detected customized gesture has at least a threshold degree of similarity to a specific gesture within the library of known gestures, and

in response to determining that the detected customized gesture has at least the threshold degree of similarity to the specific gesture within the library of known gestures, selecting the particular action based on mapping data that correlates each gesture of the library of known gestures to a respective one of the plurality of actions.

5. The computing system of claim 1 , wherein:
the customized gesture comprises the particular person picking up a phone, and
the correlated particular action comprises pausing media content being presented for display by the media player.

6. The computing system of claim 1 , wherein:
the data received from the one or more sensors represents at least one of (i) a wake word uttered by the particular person, (ii) a facial recognition of the particular person, (iii) a wake gesture performed by the particular person, (iv) a detection that the particular person is holding a remote control for the media player, or (v) a scan of a QR code presented by a display device within the viewing environment.

7. The computing system of claim 1 , wherein:
the camera is a first camera,
the first camera is mounted at a first location within the viewing environment of the media player, and
a second camera is mounted at a second location different from the first location and exterior to the viewing environment of the media player.

8. The computing system of claim 7 , wherein detecting performance of the customized gesture by the particular person comprises:
determining training data of the particular person within the viewing environment of the media player performing the customized gesture;
based on the training data and further based on pose data specifying a known orientation of the first camera, generating a classification for use by the computing system for detecting the customized gesture using the second camera;
monitoring an environment of the second camera to detect the customized gesture; and
correlating the detected customized gesture with the correlated particular action of the plurality of actions.

9. The computing system of claim 1 , wherein detecting performance of the customized gesture by the particular person comprises:
based on data received from one or more sensors in the viewing environment, detecting the particular person within one or more images of the viewing environment captured by the camera of the one or more sensors;
in response to detecting the particular person:
loading, from memory, a gesture profile associated with the detected particular person, wherein the gesture profile comprises user-specified mapping data that correlates each gesture of a library of gestures to a respective one of the plurality of actions, and
monitoring, using the camera, the viewing environment to detect performance by the person of the customized gesture; and

correlating the detected customized gesture to a particular action of the plurality of actions that the mapping data of the gesture profile correlates to the detected customized gesture.

10. The computing system of claim 1 , wherein the computing system is a controller onboard the media player.

11. A method comprising:
receiving, by a computing system and from an input device associated with the computing system, a command to map a customized gesture with a particular action of a plurality of actions that a media player is configured to perform;
in response to receiving the command, monitoring, by the computing system and using a camera, a viewing environment of the media player to detect performance by a person of the customized gesture;
in response to detecting performance of the customized gesture:
generating, by the computing system, a classification for use by the computing system for detecting the customized gesture, and
storing, by the computing system, in memory, mapping data that correlates the detected customized gesture with the particular action;

detecting multiple persons within one or more images of the viewing environment of the media player;
based on data received from one or more sensors in the viewing environment of the media player, selecting, from the multiple detected persons, a particular person to monitor for the customized gesture;
detecting performance of the customized gesture by the particular person; and
in response to detecting performance of the customized gesture by the particular person, controlling the media player to perform the correlated particular action.

12. The method of claim 11 , wherein the camera is a night vision camera.

13. The method of claim 11 , wherein the one or more sensors comprises a further camera.

14. The method of claim 11 , further comprising correlating the detected customized gesture with the correlated particular action, wherein the correlating comprises:
comparing the detected customized gesture with a library of known gestures,
determining that the detected customized gesture has at least a threshold degree of similarity to a specific gesture within the library of known gestures, and
in response to determining that the detected customized gesture has at least the threshold degree of similarity to the specific gesture within the library of known gestures, selecting the particular action based on mapping data that correlates each gesture of the library of known gestures to a respective one of the plurality of actions.

15. The method of claim 11 , wherein:
the customized gesture comprises the particular person picking up a phone, and
the correlated particular action comprises pausing media content being presented for display by the media player.

16. The method of claim 11 , wherein:
the data received from the one or more sensors repres

Assignees

• Roku Inc

Inventors

• Cutter Michael
• Ramesh Sunil
• Levitian Karina

Classifications

• H04N23/90 Primary

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06V20/52

  Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) · CPC title