Omni-spatial gesture input

Patent metadata

Field	Value
Publication number	US-9423876-B2
Application number	US-201113250532-A
Country	US
Kind code	B2
Filing date	Sep 30, 2011
Priority date	Sep 30, 2011
Publication date	Aug 23, 2016
Grant date	Aug 23, 2016

Abstract

Embodiments of the present invention relate to systems, methods and computer storage media for detecting user input in an extended interaction space of a device, such as a handheld device. The method and system allow for utilizing a first sensor of the device sensing in a positive z-axis space of the device to detect a first input, such as a user's non-device-contacting gesture. The method and system also contemplate utilizing a second sensor of the device sensing in a negative z-axis space of the device to detect a second input. Additionally, the method and system contemplate updating a user interface presented on a display in response to detecting the first input by the first sensor in the positive z-axis space and detecting the second input by the second sensor in the negative z-axis space.

First claim

The invention claimed is:

1. A method in a computing environment utilizing a processor and memory for detecting user input in both a positive z-axis space and a negative z-axis space of a device, the method comprising:
utilizing a first sensor of the device sensing first movement in a z-axis in the positive z-axis space of the device to detect a first input, wherein the first input is a non-device-contacting gesture, the positive z-axis space extending in the z-axis perpendicular to a first plane defined by an x-axis and a y-axis over a first surface of the device, the first surface comprising a display of the device;
utilizing a second sensor of the device sensing second movement in the z-axis in the negative z-axis space of the device to detect a second input, the negative z-axis space extending in the z-axis perpendicular to a second plane defined by the x-axis and the y-axis over a second surface of the device that is opposite the first surface; and
updating a user interface presented on the display of the device in response to determining a relative position of the first input in the positive z-axis space and the second input in the negative z-axis space along the z-axis, the first and second movements both being non-device contacting, the updating requiring at least both the first input from the positive z-axis space and the second input from the negative z-axis space to be sensed to be completed,
wherein the first input and the second input result from a rotational gesture approximately about an axis substantially parallel with at least one of the x-axis and the y-axis.

2. The method of claim 1 , wherein the positive z-axis space extends from a z-axis origin located at a frontal plane of the device towards a location of a user of the device.

3. The method of claim 1 , wherein the second input is a non-device-contacting gesture.

4. The method of claim 1 , wherein the first sensor is only effective for sensing an input in the positive z-axis space and the second sensor is only effective for sensing an input in the negative z-axis space.

5. The method of claim 1 , wherein the positive z-axis space and the negative z-axis space are different spaces.

6. The method of claim 1 , wherein the first input and the second input is a common gesture that is provided in both the positive z-axis space and the negative z-axis space.

7. The method of claim 1 , wherein the first input and the second input result from a rotational gesture approximately about an axis substantially parallel with a y-axis of the device.

8. The method of claim 1 , wherein the first input and the second input result from a rotational gesture approximately about an axis substantially parallel with an x-axis of the device.

9. The method of claim 1 further comprising utilizing a third sensor sensing a third input, wherein sensing the third input allows for an inference that inputs received by the first sensor or the second sensor are intended inputs.

10. The method of claim 9 , wherein the third sensor is a touch-activated sensor.

11. The method of claim 9 , wherein the third sensor is able to be activated by a first hand of the user while the first sensor and the second sensor are able to sense a second hand of the user.

12. The method of claim 9 , wherein the third sensor detects a fourth input, wherein the fourth input allows for an inference that the first input and the second input are unintentional inputs.

13. A device for detecting user input in both a positive z-axis space and a negative z-axis space of the device, the device comprising:
a device body having a front side and an opposite back side, wherein the front side is oriented toward the positive z-axis space and the back side is oriented toward the negative z-axis space;
a first sensor coupled with the device body for sensing first movement of a non-device-contacting user gesture in a z-axis in the positive z-axis space of the device, the positive z-axis space extending in the z-axis perpendicular to a first plane defined by an x-axis and a y-axis over a first surface of the device, the first surface comprising a display;
a second sensor coupled with the device body for sensing second movement of the non-device-contacting user gesture in the z-axis in the negative z-axis space of the device, the negative z-axis space extending in the z-axis perpendicular to a second plane defined by the x-axis and the y-axis over a second surface of the device that is opposite the first surface; and
a processor coupled with the device body for processing a first input from the first sensor sensing the first movement in the positive z-axis space of the device, for processing a second input from the second sensor sensing the second movement in the negative z-axis space of the device, and for identifying an instance of the non-device-contacting user gesture based on determining a relative position of the first input and the second input along the z-axis,
wherein the first input and the second input result from a rotational gesture approximately about an axis substantially parallel with at least one of an x-axis of the device and a y-axis of the device.

14. The device of claim 13 , wherein the first sensor is comprised of a camera.

15. The device of claim 13 , wherein the first sensor is comprised of a proximity sensor.

16. The device of claim 13 , wherein the first sensor and the second sensor utilize a similar sensing technology.

17. The device of claim 13 further comprising a third sensor coupled to the device body for sensing a touch-based user input.

18. The device of claim 13 , wherein the display comprises a display for displaying a user interface, wherein the user interface is updatable in response to sensing the non-device-contacting user gesture by the first sensor and the second sensor.

19. One or more computer storage devices having computer-executable instructions embodied thereon, that when executed by a computing system having a processor and memory, cause the computing system to perform a method, the method comprising:
detecting first movement in a z-axis in a positive z-axis space of the device utilizing a first depth camera, the positive z-axis space extending perpendicular to a first plane defined by an x-axis and a y-axis over a first surface of the device, the first surface comprising a display;
detecting second movement in the z-axis in a negative z-axis space of the device utilizing a second depth camera, the negative z-axis space extending in the z-axis perpendicular to a second plane defined by the x-axis and the y-axis over a second surface of the device that is opposite the first surface, the first and second movements being part of a non-device-contacting grasping user gesture comprising a first finger in the negative z-axis space and a second finger in the positive z-axis space;
identifying, with a processor, an instance of the non-device-contacting grasping user gesture based on determining the adjustment occurred using the detected first and second movements; and
adjusting a user interface on the display in response to the identifying the instance of the non-device-contacting grasping user gesture,
wherein the first movement and the second movement result from a rotational gesture approximately about an axis substantially parallel with at least one of an x-axis of the device and a y-axis of the device.

20. The one or more computer storage devices having computer-executable instructions embodied thereon of claim 19 , the method further comprising utilizing a third sensor sensin

Assignees

• Horvitz Eric
• Hinckley Kenneth P
• Benko Hrvoje
• Tan Desney S
• Microsoft Technology Licensing Llc

Inventors

• Horvitz Eric
• Hinckley Kenneth P
• Benko Hrvoje
• Tan Desney S

Classifications

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title