Head-mounted display device, program, and method for controlling head-mounted display device

Patent metadata

Field	Value
Publication number	US-10948974-B2
Application number	US-201815899395-A
Country	US
Kind code	B2
Filing date	Feb 20, 2018
Priority date	Feb 28, 2017
Publication date	Mar 16, 2021
Grant date	Mar 16, 2021

Abstract

A head-mounted display device includes: an image display unit which is mounted on a head of a user and displays an image in such a way that the user can visually recognize an external scene; a camera; a detection control unit which detects an operation with a pointing element, based on a picked-up image picked up by the camera; and an operation control unit which controls the image on the image display unit, based on the operation detected by the detection control unit. If the pointing element is detected from the picked-up image picked up by the camera, the operation control unit causes the image display unit to display a hand-finger image showing at least one of position and shape of the pointing element.

First claim

What is claimed is:

1. A head-mounted display device for use with a head of a user, the head-mounted display device comprising:
an image display that is mounted on the head of the user and displays an image;
a camera that picks up a picked-up image from an external scene;
a processor and/or circuit configured to:
detect a pointing element based on the picked-up image picked up by the camera;
cause the image display to display a display object corresponding to a position or a shape of the detected pointing element in the image, the display object being superimposed over the external scene that includes the detected pointing element, the display object being a contour line constituting a part or an entirety of a contour of the detected pointing element;
cause the image display to display a functional display image, the functional display image is the image indicating a function that executes an application; and
change a width of the contour line in the image display.

2. The head-mounted display device according to claim 1 , wherein
the image display displays the image as superimposed on the external scene visually recognized by the user,
the camera picks up an image over a range including at least a part of the external scene visually recognized by the user, and
the processor and/or circuit is configured to:
cause the display object to be displayed at a position corresponding to a position where the user visually recognizes the detected pointing element in the external scene.

3. The head-mounted display device according to claim 1 , wherein
the image display is configured to transmit an external light and is configured to adjust a transmittance of the external light transmitted through the image display, and
the processor and/or circuit is configured to:
switch between display and non-display of the display object, based on the adjusted transmittance of the external light.

4. The head-mounted display device according to claim 1 , wherein
the processor and/or circuit is configured to:
switch between display and non-display of the display object, based on a size of the image displayed by the image display.

5. The head-mounted display device according to claim 1 , wherein
the processor and/or circuit is configured to:
detect a sub-pointing element that is different from the detected pointing element, based on the picked-up image that is picked up by the camera.

6. The head-mounted display device according to claim 5 , wherein
the processor and/or circuit is further configured to:
determine a selected site based on a pointed position of the sub-pointing element on the detected pointing element or the detected display object, and
execute the application based on the selected site.

7. The head-mounted display device according to claim 1 , wherein the functional display image is displayed as superimposed on the detected pointing element and the display object.

8. The head-mounted display device according to claim 1 , wherein the display object is an image of a hand of the user picked up by the camera.

9. The head-mounted display device according to claim 6 , wherein the processor and/or circuit is configured to execute: (A) an operation mode in which the processing corresponding to the site on the detected pointing element or the detected display object selected by the operation is executed, and (B) a setting mode in which the processing corresponding to the site on the detected pointing element or the display object is set.

10. A non-transitory computer readable medium storing a program for use with a head-mounted display device having: (i) an image display which is mounted on a head of a user and that displays an image, (ii) a camera, and (iii) a processor and/or circuit, that the program causing the processor and/or circuit to execute steps comprising:
detecting a pointing element based on the picked-up image picked up by the camera;
causing the image display to display a display object corresponding to a position or a shape of the detected pointing element in the image, the display object being superimposed over the picked-up image of the external scene that includes the detected pointing element, the display object being a contour line constituting a part or an entirety of a contour of the detected pointing element;
causing the image display to display a functional display image, the functional display image indicating a function that executes an application; and
changing a width of the contour line in the image display.

11. A method for controlling a head-mounted display device that includes an image display which is mounted on a head of a user and displays an image, a camera, and a processor and/or circuit, the method comprising:
detecting a pointing element based on a picked-up image that is picked up by the camera;
causing the image display to display a display object indicating at least one of position or shape of the detected pointing element in the image, the display object being superimposed over the picked-up image of the external scene that includes the detected pointing element, the display object being a contour line constituting a part or an entirety of a contour of the detected pointing element;
causing the image display to display a functional display image, the functional display image indicating a function that executes an application; and
changing a width of the contour line in the image display.

12. The head-mounted display device according to claim 1 , wherein the functional display image is displayed around the detected pointing element or the display object.

13. A head-mounted display device for use with a head of a user, the head-mounted display device comprising:
an image display that is mounted on the head of the user and displays an image;
a camera that picks up a picked-up image from an external scene;
a processor and/or circuit configured to:
detect a pointing element based on the picked-up image picked up by the camera;
detect a sub-pointing element that is different from the detected pointing element form on the picked-up image;
cause the image display to display a display object corresponding to a position or a shape of the detected pointing element in the image, the display object being superimposed over the external scene that includes the detected pointing element, the display object being a first contour line that is a part or an entirety of a contour of the detected pointing element;
cause the image display to display a functional display image, the functional display image is the image indicating a function that executes an application;
cause the image display to display a sub-display object, the sub-display object is a second contour line that is a part or entirety of a contour of the sub-pointing element; and
cause the image display to display a width of the first contour line with different from a width of the second contour line.

Assignees

• Seiko Epson Corp

Inventors

• Kunitomo Yuichi

Classifications

• G02B27/0172 Primary

  characterised by optical features · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title