Apparatus and method pertaining to the use of a plurality of 3D gesture sensors to detect 3D gestures

Patent metadata

Field	Value
Publication number	US-9817565-B2
Application number	US-201313948283-A
Country	US
Kind code	B2
Filing date	Jul 23, 2013
Priority date	Jul 23, 2013
Publication date	Nov 14, 2017
Grant date	Nov 14, 2017

Abstract

A device having at least two 3-dimensional gesture sensors that employ differing gesture-sensing modalities as compared to one another further includes a control circuit that operably couples to both of these 3-dimensional gesture sensors and employs both to detect three-dimensional gestures. By one approach the control circuit employs both sensors in a temporally-overlapping manner to reliably and accurately detect the 3D gesture. As another illustrative example, the control circuit may employ different sensors during different portions of a given 3D gesture to detect those corresponding portions of the 3D gesture.

First claim

What is claimed is:

1. An apparatus comprising:
a processor and memory running an application using one or more gestures as inputs;
a touch-sensitive display for sensing two-dimensional gestures in a two-dimensional plane;
a first non-contact three-dimensional gesture sensor for sensing a user's hand movement in front of and relative to the touch-sensitive display;
at least a second non-contact three-dimensional gesture sensor for sensing the user's hand movement in front of and relative to the touch-sensitive display, the first and second non-contact three-dimensional gesture sensors employing differing gesture-sensing modalities;
a control circuit operably coupled to the first and second non-contact three-dimensional gesture sensors and configured to switch between the first non-contact three-dimensional gesture sensor and the at least a second non-contact three-dimensional gesture sensor while in the process of detecting a current three-dimensional gesture of the user's hand movement,
wherein the switch is based on each of
remaining predictable portions of the currently detected three-dimensional gesture, and
a combination of sensing accuracy, power consumption and computational intensity.

2. The apparatus of claim 1 wherein the gesture-sensing modalities include at least one gesture-sensing modality selected from a group comprising a camera-based modality, a capacitive-based modality, an infrared light-based modality, an ambient light-based modality, and an ultrasonic-based modality.

3. The apparatus of claim 1 wherein the control circuit is configured to employ both the first and second non-contact three-dimensional gesture sensors to detect a three-dimensional gesture by, at least in part, testing each of the first and second non-contact three-dimensional gesture sensors with respect to a current three-dimensional gesture to identify a best non-contact three-dimensional gesture sensor to employ to detect the current three-dimensional gesture.

4. The apparatus of claim 1 wherein the control circuit is configured to employ both the first and second non-contact three-dimensional gesture sensors to detect a three-dimensional gesture by, at least in part, using each of the first and second non-contact three-dimensional gesture sensors to detect a portion of a current three-dimensional gesture.

5. The apparatus of claim 4 wherein the control circuit is configured to use each of the first and second non-contact three-dimensional gesture sensors to detect a portion of a current three-dimensional gesture by, at least in part, using at least one of the first and second non-contact three-dimensional gesture sensors to detect a portion, but only a portion, of the current three-dimensional gesture.

6. The apparatus of claim 1 wherein the control circuit is configured to employ both the first and second non-contact three-dimensional gesture sensors to detect a three-dimensional gesture by, at least in part, determining which of the first and second non-contact three-dimensional gesture sensors is best suited to detect at least certain portions of a predicted three-dimensional gesture.

7. The apparatus of claim 6 wherein the control circuit is configured to determine which of the first and second non-contact three-dimensional gesture sensors is best suited as a function, at least in part, of both sensing accuracy and power consumption.

8. A method comprising on an electronic device by a control circuit that is operably coupled to
a processor and memory running an application using one or more gestures as inputs;
a touch-sensitive display for sensing two-dimensional gestures in a two-dimensional plane;
a first non-contact three-dimensional gesture sensor for sensing a user's hand movement in front of and relative to the touch-sensitive display and at least a second non-contact three-dimensional gesture sensor for sensing the user's hand movement in front of and relative to the touch-sensitive display, the first and second non-contact three-dimensional gesture sensors employing differing gesture-sensing modalities; and
switching between, based upon presets in the application, the first non-contact three-dimensional gesture sensor to detect a first portion of a current three-dimensional gesture of the user's hand movement and second non-contact three-dimensional gesture sensor to detect a second portion of the current three-dimensional gesture, wherein the switching is based on each of
remaining predictable portions of the currently detected three-dimensional gesture, and
a combination of sensing accuracy, power consumption and computational intensity.

9. The method of claim 8 wherein the gesture-sensing modalities include at least one gesture-sensing modality selected from a group comprising a camera-based modality, a capacitive-based modality, an infrared light-based modality, an ambient light-based modality, and an ultrasonic-based modality.

10. The method of claim 8 wherein employing both the first and second non-contact three-dimensional gesture sensors to detect a three-dimensional gesture comprises, at least in part, testing each of the first and second non-contact three-dimensional gesture sensors with respect to a current three-dimensional gesture to identify a best non-contact three-dimensional gesture sensor to employ to detect the current three-dimensional gesture.

11. The method of claim 8 wherein employing both the first and second non-contact three-dimensional gesture sensors to detect a three-dimensional gesture comprises, at least in part, using each of the first and second non-contact three-dimensional gesture sensors to detect a portion of a current three-dimensional gesture.

12. The method of claim 11 wherein using each of the first and second non-contact three-dimensional gesture sensors to detect a portion of a current three-dimensional gesture comprises, at least in part, using at least one of the first and second non-contact three-dimensional gesture sensors to detect a portion, but only a portion, of the current three-dimensional gesture.

13. The method of claim 8 wherein employing both the first and second non-contact three-dimensional gesture sensors to detect a three-dimensional gesture comprises, at least in part, determining which of the first and second non-contact three-dimensional gesture sensors is best suited to detect at least certain portions of a predicted three-dimensional gesture.

14. The method of claim 13 wherein determining which of the first and second non-contact three-dimensional gesture sensors is best suited is determined as a function, at least in part, of both sensing accuracy and power consumption.

15. A non-transitory digital memory having computer instructions stored therein, the computer instructions, when executed by a processor in an apparatus, serving to cause the processor to employ
an application executed by the processor using one or more gestures as inputs;
a touch-sensitive display for sensing two-dimensional gestures in a two-dimensional plane;
a first non-contact three-dimensional gesture sensor for sensing a user's hand movement in front of and relative to the touch-sensitive display and second non-contact three-dimensional gesture sensor for sensing the user's hand movement in front of and relative to the touch-sensitive display, and based upon presets in the application, the first non-contact three-dimensional gesture sensor to detect a first portion of a three-dimensional gesture and second non-contact three-dimensional gesture sensor to detect a second portion of the three-dimensional gesture, the first and second non-contact three-dimensional

Assignees

• Blackberry Ltd

Inventors

• Idzik Jacek S
• Mankowski Peter
• Mercea Cornel

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0487 Primary

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title