Eyewear-type terminal and method for controlling the same

Patent metadata

Field	Value
Publication number	US-10061391-B2
Application number	US-201514951318-A
Country	US
Kind code	B2
Filing date	Nov 24, 2015
Priority date	May 26, 2015
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

Provided is a eyewear-type terminal capable of communicating with a drone. The eyewear-type terminal includes a communication unit configured to communicate with a drone, a display configured to output visual information received from the drone, an optical unit comprising at least one lens and configured to form an image of the visual information in an external virtual region such that the image is visible only to a user wearing the terminal, a sensor configured to sense a user gesture applied with respect to the image and a controller configured to control the drone based on the sensed user gesture.

First claim

What is claimed is:

1. An eyewear-type terminal comprising:
a communication unit configured to communicate with a drone;
a display configured to output visual information received from the drone;
an optical unit comprising at least one lens and configured to form a virtual image of the visual information at an external virtual region that is formed outside of the terminal such that the virtual image is visible only to a user wearing the terminal;
a sensor configured to sense a user gesture applied with respect to the virtual image; and
a controller configured to:
control the drone based on the sensed user gesture; and
control movement of the drone based on the sensed user gesture that is applied subsequently to a gesture of grabbing the virtual image such that:
the drone is moved when a manner of the gesture of grabbing the virtual image is a first manner; and
the drone is rotated when the manner of the gesture of grabbing the virtual image is a second manner.

2. The terminal of claim 1 , wherein the controller is further configured to control at least one of a movement of the drone or a camera of the drone in response to a preset user gesture applied with respect to the virtual image.

3. The terminal of claim 2 , wherein the controller is further configured to:
control the movement of the drone when the preset user gesture is a first user gesture; and
control the camera when the preset user gesture is a second user gesture.

4. The terminal of claim 3 , wherein:
the first user gesture is a gesture that is applied subsequently to a gesture of grabbing the virtual image; and
the controller is further configured to cause the drone to move in response to the subsequently applied gesture.

5. The terminal of claim 4 , wherein when the first user gesture is applied in a state in which the virtual image is displayed in a first display manner, the virtual image is displayed in a second display manner.

6. The terminal of claim 5 , wherein the virtual image displayed in the second display manner is restored to be displayed in the first display manner when the first user gesture is released.

7. The terminal of claim 3 , wherein:
the second user gesture comprises a pinch-in gesture or a pinch-out gesture applied with respect to the virtual image; and
the controller is further configured to control the camera to execute a zoom-out function or a zoom-in function in response to the pinch-in gesture or the pinch-out gesture.

8. The terminal of claim 7 , wherein the controller is further configured to cause the drone to move when the pinch-out gesture is applied to be moved out of the virtual image.

9. The terminal of claim 1 , further comprising a camera, wherein:
the controller is further configured to cause the display to display a preview image received via the camera; and
the optical unit is further configured to form an image of the preview image in addition to the virtual image of the visual information.

10. The terminal of claim 9 , wherein the image of the preview image and the virtual image of the visual information are formed to be spaced apart from one another.

11. The terminal of claim 9 , wherein when a preset type of gesture is applied to at least one of the image of the preview image or the virtual image of the visual information, the controller is further configured to cause the at least one of the image of the preview image or the virtual image of the visual information to move.

12. The terminal of claim 11 , wherein when the image of the preview image and the virtual image of the visual information come into contact with each other due to the movement, the controller is further configured to capture the image of the preview image and the virtual image of the visual information.

13. An eyewear-type terminal comprising:
a communication unit configured to communicate with a drone;
a display configured to output visual information received from the drone;
an optical unit comprising at least one lens and configured to form a virtual image of the visual information at an external virtual region such that the virtual image is visible only to a user wearing the terminal;
a sensor configured to sense a user gesture applied with respect to the virtual image; and
a controller configured to:
control movement of the drone when the sensed user gesture is a first user gesture that is applied subsequently to a gesture of grabbing the virtual image, causing the drone to move in response to the subsequently applied gesture, wherein the drone is moved in a different manner according to a manner of the gesture of grabbing the virtual image; and
control a camera of the drone when the sensed user gesture is a second user gesture.

14. A method for controlling an eyewear-type terminal, the method comprising:
outputting visual information received from a drone via a display of the terminal;
forming a virtual image of the visual information, by an optical unit comprising at least one lens, in an external virtual region that is formed outside of the terminal such that the virtual image is visible only to a user wearing the terminal;
controlling the drone based on a user gesture applied with respect to the virtual image; and
controlling movement of the drone based on the user gesture that is applied subsequently to a gesture of grabbing the virtual image such that:
the drone is moved when a manner of the gesture of grabbing the virtual image is a first manner; and
the drone is rotated when the manner of the gesture of grabbing the virtual image is a second manner.

15. The method of claim 14 , wherein:
the controlling of the drone comprises controlling at least one of a movement of the drone or a camera of the drone in response to a preset user gesture applied with respect to the virtual image;
the movement of the drone is controlled when the preset user gesture is a first user gesture; and
the camera is controlled when the preset user gesture is a second user gesture.

16. The method of claim 15 , wherein:
the first user gesture is a gesture that is applied subsequently to the gesture of grabbing the virtual image; and
the controller is further configured to cause the drone to move in response to the subsequently applied gesture.

17. The method of claim 15 , wherein:
the second user gesture comprises a pinch-in gesture or a pinch-out gesture applied with respect to the virtual image; and
the controller is further configured to control the camera to execute a zoom-out function or a zoom-in function in response to the pinch-in gesture or the pinch-out gesture.

18. The method of claim 14 , further comprising:
forming an image of a preview image received via a camera of the terminal in addition to the virtual image of the visual information,
wherein at least one of the image of the preview image or the virtual image of the visual information is moved in response to a preset type of gesture applied to the at least one of the image of the preview image or the virtual image of the visual information.

19. The method of claim 18 , further comprising:
capturing the image of preview image and the virtual image of the visual information when the image of the preview image and the virtual image of the visual information come into contact with each other due to the movement.

Assignees

• Lg Electronics Inc

Inventors

• Kwon Chohee
• Han Dongsu
• Lim Gukchan

Classifications

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G02B27/0179

  Display position adjusting means not related to the information to be displayed · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title