Zoom-based gesture user interface

Patent metadata

Field	Value
Publication number	US-9836201-B2
Application number	US-201414485840-A
Country	US
Kind code	B2
Filing date	Sep 15, 2014
Priority date	Jul 5, 2011
Publication date	Dec 5, 2017
Grant date	Dec 5, 2017

Abstract

A user interface method, including presenting by a computer executing a user interface, multiple interactive items on a display. A first sequence of images is captured indicating a position in space of a hand of a user in proximity to the display, and responsively to the position, one of the interactive items is associated with the hand. After associating the item, a second sequence of images is captured indicating a movement of the hand, and responsively to the movement, a size of the one of the items is changed on the display.

First claim

We claim:

1. A user interface method, comprising:
presenting by a computer executing a user interface, multiple interactive items on a display;
capturing a first sequence of images indicating a position in space of a hand of a user in proximity to the display;
responsively to the position, associating a given interactive item on the display with the hand;
after associating the given item, capturing a second sequence of images indicating a movement of the hand away from the display; and
responsively to the movement away from the display, changing a size of the given item on the display.

2. The method according to claim 1 , wherein the given interactive item is enlarged in response to the movement of the hand comprising motion away from the display.

3. The method according to claim 1 , wherein the given interactive item is associated with a function of the computer, and wherein the method comprises activating the function when the given interactive item has been enlarged, in response to the movement, to at least a predetermined threshold size.

4. The method according to claim 3 , wherein activating the function comprises playing a media item associated with the given interactive item.

5. The method according to claim 3 , wherein activating the function comprises executing a software application.

6. The method according to claim 3 , wherein activating the function comprises displaying an image over a full area of the display.

7. The method according to claim 1 , wherein associating the given interactive item comprises choosing the interactive item that is presented at a center of the display.

8. The method according to claim 1 , wherein associating the given interactive item comprises detecting, in the first sequence of images, a predetermined gesture of the hand, and initiating association of the given interactive item with the hand responsively to the gesture.

9. The method according to claim 1 , wherein at least some of the interactive items contain sub-items, which are not displayed as long as the interactive items presented on the display are smaller than a predetermined size, and wherein enlarging the given interactive item comprises displaying the sub-items contained by the given interactive item when the given interactive item is enlarged to at least the predetermined size.

10. The method according to claim 9 , wherein the given interactive item is enlarged in response the movement comprising significant motion away from the display.

11. The method according to claim 1 , wherein sensing the position comprises generating a three-dimensional (3D) map of at least a part of body of the user, including the hand.

12. The method according to claim 1 , wherein associating the item with the hand comprises identifying the item based on a gaze of the user.

13. The method according to claim 1 , wherein associating the item with the hand comprises identifying the item based on a pointing gesture performed the user.

14. The method according to claim 1 , wherein presenting the multiple interactive items comprises organizing the multiple interactive items as a grid on the display.

15. The method according to claim 1 , wherein changing the size of the one of the items comprises enlarging the one of the items on the display.

16. A user interface method, comprising:
presenting by a computer executing a user interface, multiple interactive items on a display;
capturing a first sequence of images indicating a position in space of a hand of a user in proximity to the display;
capturing a second sequence of images indicating a movement of the hand transverse to the display in a first direction; and
responsively to the movement, panning the interactive items on the display in a second direction, which is different from the first direction.

17. The method according to claim 16 , wherein the second direction comprises a direction opposite to the first direction.

18. A user interface method, comprising:
presenting by a computer executing a user interface, multiple interactive items on a display;
receiving, from a handheld remote control device coupled to the computer, a first signal indicating a position in space of a hand of a user in proximity to the handheld remote control device;
responsively to the position, associating one of the interactive items with the hand;
after associating the item, receiving from the handheld remote control device a second signal indicating a movement of the hand relative to the handheld remote control device; and
responsively to the movement, changing a size of the one of the interactive items on the display.

Assignees

• Apple Inc

Inventors

• Frey Martin
• Hauer Marcus
• Buzzini Dario
• Schaefer Philipp
• Berenson Adi
• Galor Micha
• Metuki Nili
• Shpunt Alexander

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0485

  Scrolling or panning · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title