Method and device for gesture recognition diagnostics for device orientation

Patent metadata

Field	Value
Publication number	US-8941587-B2
Application number	US-201113220361-A
Country	US
Kind code	B2
Filing date	Aug 29, 2011
Priority date	Apr 13, 2011
Publication date	Jan 27, 2015
Grant date	Jan 27, 2015

Abstract

Systems, circuits, and devices for recognizing gestures are discussed. A mobile device includes a housing, an orientation sensor, a camera implemented on the housing, a memory for storing a lookup table comprising multiple gestures and corresponding commands, and a controller coupled to the orientation sensor, the camera, and the memory. The controller is configured to generate trace data corresponding to a gesture captured by the camera, wherein x, y, and z coordinates of the trace data are applied according to an orientation of the housing during the gesture. The controller is also configured to determine an orientation angle of the housing detected by the orientation sensor. The controller is further configured to recognize the gesture through accessing the lookup table based on the trace data and the orientation angle of the housing.

First claim

What is claimed is:

1. An image display device, comprising:
a housing;
an orientation sensor configured to determine an orientation angle of the housing;
a display configured to display an image;
a camera configured to capture a user gesture in a field of view of the camera; and
a controller operatively coupled to the orientation sensor, the camera, and the display, the controller configured to:
generate trace data corresponding to the gesture captured by the camera in response to a receipt of the user gesture;
convert the user gesture to an execution command based on the orientation angle of the housing and the trace data; and
execute the execution command upon recognition of the user gesture as the execution command.

2. The image display device of claim 1 , wherein the image display device is a mobile device.

3. The image display device of claim 2 , wherein the mobile device includes a transceiver configured to communicate to another device.

4. The image display device of claim 1 , wherein the trace data includes at least two of x, y, and z orthogonal coordinates relative to one of the camera and the orientation angle.

5. The image display device of claim 4 , wherein the controller is configured to convert the captured user gesture into the trace data by
capturing, by the camera, a plurality of frames from an initial instance of the gesture to a final instance of the gesture; and
determining the at least two of the x, y, and z orthogonal coordinates of the trace data from the plurality of frames.

6. The image display device of claim 4 , wherein the controller is configured to recognize the user gesture based on the trace data.

7. The image display device of claim 6 , further comprising:
a memory operatively coupled to the controller and configured to store gesture information and a gesture-specific execution command that corresponds to the orientation angle,
wherein the controller is configured to access the gesture information and convert the recognized user gesture to the gesture-specific execution command that corresponds to the orientation angle.

8. The image display device of claim 7 , wherein the controller is configured to convert the recognized user gesture into the gesture-specific execution command by determining that a pattern of the trace data substantially matches a pattern in the memory.

9. The image display device of claim 1 , wherein the orientation sensor is an accelerometer, a gyroscope, or a magnetometer.

10. The image display device of claim 1 , wherein the camera is a depth-aware camera, a stereo camera, or a digital camera.

11. The image display device of claim 1 , wherein the controller is configured to control the display to display the image in one of a landscape orientation and a portrait orientation when the orientation angle of the housing is less than a threshold angle and to display the image in another of the landscape orientation and the portrait orientation when the orientation angle of the housing is greater than the threshold angle.

12. The image display device of claim 1 , wherein the controller is configured to obtain the orientation angle by measuring an angle between a straight line crossing the camera in a lengthwise direction of the housing and a straight line crossing the camera in a widthwise direction of the housing.

13. An image display device, comprising:
a housing;
an orientation sensor configured to determine an orientation of the housing relative to a predetermined coordinate system;
a display configured to display an image;
a camera configured to capture an object in a field of view of the camera; and
a controller coupled to the display, the orientation sensor and the camera, the controller configured to:
display a pointer on the display at a specific location relative to the displayed image, the specific location corresponding to a first orientation of the housing and a detected location of the object, and
when the orientation of the housing is changed to a second orientation, reorient the image according to the second orientation while maintaining the pointer at the specific location relative to the displayed image.

14. The image display device of claim 13 ,
wherein the controller is configured to identify coordinates of the pointer when the housing is in the first orientation, and
wherein the controller is configured to maintain the pointer at the specific location relative to the displayed image when the housing is in the second orientation based on the coordinates.

15. The image display device of claim 13 ,
wherein the controller is configured to determine a first distance between the pointer and a first part of the image and a second distance between the pointer and a second part of the image when the housing is in the first orientation, and
wherein the controller is configured to maintain the pointer at the specific location relative to the displayed image when the housing is in the second orientation based on the first and second distances.

16. The image display device of claim 13 , wherein the first orientation is one of a landscape orientation and a portrait orientation, and the second orientation is another of the landscape orientation and the portrait orientation.

17. The image display device of claim 13 , wherein the controller is configured to detect the change of the housing to the second orientation when tilt data generated by the orientation sensor is greater than a threshold angle.

18. An image display device, comprising:
a housing;
an orientation sensor configured to determine an orientation of the housing;
a display configured to display an image;
a camera configured to capture an object in a field of view of the camera; and
a controller coupled to the display, the orientation sensor and the camera, the controller configured to:
display a pointer on the display at a specific location, the specific location corresponding to the housing and a detected location of the object,
reorient the image according to a second orientation when the orientation of the housing is changed to the second orientation, and
display a pointer on the display at the specific location, the specific location corresponding to the housing and the detected location of the object regardless of the orientation of the housing.

Assignees

• Kim Sangki
• Yeo Junhee
• Lg Electronics Inc

Inventors

• Kim Sangki
• Yeo Junhee

Classifications

• G06F2340/0442

  Physics · mapped topic

• G06F2340/0492

  Physics · mapped topic

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F2200/1637

  Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer · CPC title

• G06F2200/1614

  Image rotation following screen orientation, e.g. switching from landscape to portrait mode · CPC title