Head-mounted display device and method of controlling head-mounted display device

Patent metadata

Field	Value
Publication number	US-9335547-B2
Application number	US-201414212225-A
Country	US
Kind code	B2
Filing date	Mar 14, 2014
Priority date	Mar 25, 2013
Publication date	May 10, 2016
Grant date	May 10, 2016

Abstract

A transmissive head-mounted display device includes an image display unit that generates image light representing an image, allows a user to visually recognize the image light, and transmits outside scenery when worn on a head of a user, a detection unit that detects at least one of a line-of-sight direction of the user and a shielding object that is located at a predetermined distance from an eye of the user and shields a line of sight of the user, and a control unit that performs control of the image light based on the detected detection result.

First claim

What is claimed is:

1. A transmissive head-mounted display device comprising:
an image display unit that generates image light representing an image, allows a user to visually recognize the image light, and transmits outside scenery when worn on a head of a user;
a detection unit that detects at least one of a line-of-sight direction of the user and a shielding object that is located at a predetermined distance from an eye of the user and shields a line of sight of the user; and
a control unit that performs control of the image light based on the detected detection result,
wherein
the image display unit allows the user to visually recognize a plurality of selectable objects and a pointer indicating a position corresponding to the detected line-of-sight direction as the image light,
the control unit performs control on the head-mounted display device in response to a positional relationship between the plurality of selectable objects and the pointer,
the detection unit detects the shielding object in the line-of-sight direction,
the control unit performs, as the control, selection of one of the selectable objects based on the positional relationship, and performs determination of the selected object based on the detection result of the shielding object,
the detection unit detects a right eye front index value and a left eye front index value indicating at least one of colors and illuminance in a range from the respective right eye and left eye of the user to the predetermined distance, and
the control unit performs control in response to the positional relationship and the detected right eye front index value and left eye front index value.

2. The head-mounted display device according to claim 1 , wherein the detection unit detects an eyelid status of the user as the shielding object, and
the control unit performs control in response to the detected eyelid status and the positional relationship.

3. The head-mounted display device according to claim 1 , wherein the image display unit sets an area around the detected line-of-sight direction and images the outside scenery within the area based on the detection result of the shielding object.

4. The head-mounted display device according to claim 1 , further comprising:
an imaging unit that images the outside scenery; and
an image recognition unit that recognizes a specific object contained in the imaged outside scenery and existing in a position farther than the predetermined distance,
wherein the image display unit sets an area around the recognized specific object and images the outside scenery within the area based on the recognized specific object.

5. The head-mounted display device according to claim 1 , further comprising an orientation detection unit that detects an orientation of the head of the user,
wherein the image display unit sets an area around a line-of-sight based on the detected orientation of the head of the user and images the outside scenery within the area based on the detected orientation of the head.

6. The head-mounted display device according to claim 1 , further comprising an imaging unit that images the outside scenery,
wherein the control unit performs operation of the imaging unit based on the detection result of the shielding object as the control, and
the image display unit generates area image light as image light representing outside scenery within an imaging area.

7. The head-mounted display device according to claim 6 , further comprising an image recognition unit that recognizes a specific object contained in the imaged outside scenery based on the operation,
wherein, when the specific object is recognized in the imaged outside scenery, the control unit sets a partial area of the imaged outside scenery based on the specific object, and
the outside scenery image light is image light representing the set partial area.

8. The head-mounted display device according to claim 7 , wherein, when the specific object is not recognized in the imaged outside scenery, the control unit estimates a direction from the imaging unit to the specific object based on change in position of the specific object when the specific object is recognized in the imaged outside scenery, and
the imaging unit images the estimated direction.

9. The head-mounted display device according to claim 6 , further comprising an orientation detection unit that detects an orientation of the head of the user with respect to the direction of gravity force,
wherein, when an angle formed by the detected orientation of the head and the direction of gravity force is equal to or more than a predetermined value, the image display unit generates the area image light as image light representing at least a partial area of outside scenery imaged with an axis in parallel to the horizontal plane orthogonal to the direction of gravity force as an optical axis based on the angle.

10. The head-mounted display device according to claim 1 , wherein the image display unit generates line-of-sight position image light as image light representing a position corresponding to the line-of-sight direction with respect to the imaged outside scenery and allows the user to visually recognize the line-of-sight position image light,
the control unit sets a partial area of the imaged outside scenery based on the detected line-of-sight direction and the combination, and
the area image light is image light representing the set partial area.

11. The head-mounted display device according to claim 10 , wherein the operation includes adjustment of a focal length of a lens of the imaging unit, and
regarding the area image light, when the focal length is within the predetermined distance, the area image light is generated in a peripheral part in the partial area, and, when the focal length is beyond the predetermined distance, outside scenery image light representing an outside scenery image in the partial area is generated as the area image light.

12. The head-mounted display device according to claim 1 , wherein the detection unit detects fluctuations in line of sight when the user fixates a specific point, and
the control unit performs the control based on the fluctuations in line of sight.

13. The head-mounted display device according to claim 12 , wherein, when an amplitude of the fluctuations in line of sight is less than a threshold value, the control unit determines that the user fixates the specific point.

14. The head-mounted display device according to claim 13 , wherein
when determining that the user fixates the specific point, the control unit sets the object to be selectable.

15. The head-mounted display device according to claim 12 , wherein, when the line-of-sight direction falls within a predetermined range around the specific point in a predetermined period, the control unit determines that the user fixates the specific point.

16. The head-mounted display device according to claim 15 , wherein the image display unit allows a user to visually recognize image light representing the predetermined range as the image light.

17. The head-mounted display device according to claim 15 , further comprising a sound output unit,
wherein, when the line-of-sight direction falls within the predetermined range in the predetermined period, the control unit allows the sound output unit to output specific sound.

18. The head-mounted display device according to claim 1 , wherein the detection unit detects a palm of the user as the shielding object.

19. The head-mounted display device according to claim

Assignees

• Seiko Epson Corp

Inventors

• Takano Masahide
• Kimura Fusashi

Classifications

• H04M1/724097

  Worn on the head · CPC title

• H04M1/72412

  using two-way short-range wireless interfaces · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title