Apparatus and method for inputting command using gesture

Patent metadata

Field	Value
Publication number	US-9836127-B2
Application number	US-201114000699-A
Country	US
Kind code	B2
Filing date	Apr 22, 2011
Priority date	Feb 23, 2011
Publication date	Dec 5, 2017
Grant date	Dec 5, 2017

Abstract

Disclosure is a method of inputting commands into displays such as TVs or image processing devices. User's hands have been photographed through a camera to recognize the motion of the user's hands, so that commands are input according to the motion of user's hands instead of conventional input devices such as a mouse and a keyboard.

First claim

The invention claimed is:

1. An input apparatus using a gesture comprising:
at least one camera photographing a gesture image of a user's hands introduced into a predetermined detection region near the input apparatus;
a controller displaying content, a virtual image of a device to which a gesture of the user is applied, and a user gesture image that has been photographed in a gesture recognition mode, and executing a command according to the gesture of the user;
a storage unit storing a command code used to run an operation of the controller;
a micro-phone receiving a voice of the user; and
a display outputting the images;
wherein, when a keyboard input mode is executed according to the voice received through the micro-phone, a keyboard input window is displayed, wherein a keyboard virtual image and input values are displayed on the keyboard input window;
wherein, in the keyboard mode, the controller is configured to:
recognize a finger position on the user gesture image based on a position of a user's finger in the detection region;
compare the finger position with a position of the keyboard virtual image to recognize the finger position on the keyboard virtual image;
display a feedback image corresponding to the finger position on the keyboard virtual image;
set points to positions of finger joints of the finger position;
detect a key input gesture position on the keyboard virtual image when a key input gesture is detected by recognizing a movement of each of the points; and
display a key image corresponding to the key input gesture position on the keyboard virtual image;
wherein the user's finger does not touch any real input device or the virtual image of the device;
wherein, when a mouse input mode is executed according to the voice received through the micro-phone, a virtual mouse image is displayed;
wherein, when the user makes a gesture to grip the virtual mouse image in the mouse mode, a hand image corresponding to the user's hand grips the virtual mouse image, and, in this state, when the user moves the user's hand, the virtual mouse image moves accordingly;
wherein, when a phone mode is executed according to the voice received through the micro-phone, an input window to display a key pad and phone numbers, which have been input, is displayed; and
wherein all of the key pad and the input window are displayed as semi-transparent images.

2. The input apparatus of claim 1 , wherein the controller activates or changes the gesture recognition mode according to the voice received through the micro-phone.

3. The input apparatus of claim 1 , wherein the camera is an infrared camera or a visible light camera.

4. The input apparatus of claim 1 , wherein the device to which the gesture of the user is applied includes at least one of a keyboard, a mouse, and a phone number key pad.

5. The input apparatus of claim 1 , wherein the feedback image is a focus-on image corresponding to the finger position when the key input gesture is not yet detected.

6. The input apparatus of claim 1 , wherein the feedback image comprises a plurality of focus-on images displaying a plurality of positions of a plurality of fingers.

7. The input apparatus of claim 1 , wherein the at least one camera comprises a plurality of cameras installed at a predetermined position of a display.

8. The input apparatus of claim 1 , wherein there is no real input device in the predetermined detection region, and
wherein the at least one camera comprises two cameras to serve as a stereo camera.

9. The input apparatus of claim 1 , wherein, in the mouse input mode, when the user makes a gesture to click a right or left button of the mouse, the hand image in which the user's hand performs the right-click or left-click of the mouse is displayed, and the operation corresponding to the click is executed.

10. An input method using a gesture of a user, the input method comprising:
photographing a gesture of a user introduced into a predetermined detection region;
simultaneously displaying content and a virtual image of a device to which the gesture of the user is applied in a gesture recognition mode;
displaying a photographed image of the gesture of the user;
displaying a feedback image according to the gesture of the user;
displaying a key image according to the gesture of the user; and
executing a command according to the gesture of the user;
wherein, when a keyboard input mode is executed according to a voice received through a micro-phone, a keyboard input window is displayed, wherein a keyboard virtual image and input values are displayed on the keyboard input window;
wherein, in the keyboard mode, displaying the feedback image comprises:
recognizing a finger position on the photographed image based on a position of a user's finger in the detection region;
comparing the finger position with a position of the keyboard virtual image to recognize the finger position on the keyboard virtual image;
displaying the feedback image corresponding to the finger position on the keyboard virtual image; and
setting points to positions of finger joints of the finger position;
wherein displaying the key image comprises:
detecting a key input gesture position on the keyboard virtual image when a key input gesture is detected by recognizing a movement of each of the points; and
displaying the key image corresponding to the key input gesture position on the keyboard virtual image;
wherein the user's finger does not touch any real input device or the virtual image of the device;
wherein, when a mouse input mode is executed according to the voice received through the micro-phone, a virtual mouse image is displayed;
wherein, when the user makes a gesture to grip the virtual mouse image in the mouse mode, a hand image corresponding to the user's hand grips the virtual mouse image, and, in this state, when the user moves the user's hand, the virtual mouse image moves accordingly;
wherein, when a phone mode is executed according to the voice received through the micro-phone, an input window to display a key pad and phone numbers, which have been input, is displayed; and
wherein all of the key pad and the input window are displayed as semi-transparent images.

11. The input method of claim 10 , wherein the device to which the gesture of the user is applied includes at least one of a keyboard, a mouse, and a phone number key pad.

12. The input method of claim 10 , wherein the feedback image is a focus-on image corresponding to the finger position when the key input gesture is not yet detected.

13. The input method of claim 10 , wherein the feedback image comprises a plurality of focus-on images displaying a plurality of positions of a plurality of fingers.

14. The input method of claim 10 , wherein photographing the gesture of the user is performed by at least one camera.

15. The input method of claim 14 , wherein the at least one camera comprises a plurality of cameras installed at a predetermined position of a display.

16. The input apparatus of claim 10 , wherein there is no real input device in the predetermined detection region, and
wherein the at least one camera comprises two cameras to serve as a stereo camera.

17. The input method of claim 10 , wherein, in the mouse input mode, when the user makes a gesture to click a right or left button of the mouse, the hand image in which the user's hand performs the right-click or left-click of the mouse is displayed, and the operation corresponding to the click is executed.

Assignees

• Jung Woo Kil
• Lg Innotek Co Ltd

Inventors

• Jung Woo Kil

Classifications

• G06F3/0425

  using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title