Eye tracking with mobile device in a head-mounted display

Patent metadata

Field	Value
Publication number	US-9791924-B2
Application number	US-201514711606-A
Country	US
Kind code	B2
Filing date	May 13, 2015
Priority date	Dec 23, 2014
Publication date	Oct 17, 2017
Grant date	Oct 17, 2017

Abstract

A head-mount display (HMD) and variations thereof are described. An HMD may include a mobile device having a display unit, a camera, a light source and a processing unit that controls operations of the display unit, camera and light source. The processing unit receives data associated with one or more optical images captured by the camera, and renders a visual image displayable by the display unit. The HMD may also include an eyewear piece having a holder, one or more lenses and a reflective unit. The holder is wearable by a user and holds the mobile device in front of eyes of the user. The user can view the display unit through the one or more lenses. The reflective unit reflects an image of at least an eye of the user. The camera is oriented to capture the reflected image of the eye through the reflective unit.

First claim

What is claimed is:

1. A head-mounted display, comprising:
an eyewear piece comprising:
a holder that is wearable by a user and configured to hold a mobile device in front of eyes of the user;
one or more lenses through which the mobile device is viewable by the user; and
a reflective unit configured to reflect double images of at least a first eye of the eyes of the user,
wherein the reflective unit comprises a first mirror, a second mirror and a hot mirror arranged to have the first mirror reflecting a first image of the first eye to a camera on the mobile device and the second mirror reflecting a second image of the first eye to the camera different than the first image such that the camera receives the double images of the first eye,
wherein the second mirror is disposed on the holder and facing toward the mobile device and away from the user, and
wherein the second image is reflected by the hot mirror to the second mirror, and further reflected by the second mirror to the camera.

2. The head-mounted display of claim 1 , wherein the holder comprises a pair of goggles configured to seal off a space between the eyewear piece and a face of the user to prevent an ambient light from entering the space.

3. The head-mounted display of claim 1 , wherein the reflective unit is configured to allow an infrared light to pass through and reflect a visible light or alternatively configured to allow the visible light to pass through and reflect the infrared light.

4. The head-mounted display of claim 1 , further comprising:
the mobile device comprising:
a display unit;
the camera configured to capture one or more optical images, the camera oriented to capture the images of the first eye reflected by the reflective unit;
a light source configured to emit a light; and
a processing unit configured to control operations of the display unit, the camera and the light source, the processing unit also configured to receive data associated with the captured one or more optical images from the camera, the processing unit further configured to render a visual image displayable by the display unit.

5. The head-mounted display of claim 4 , wherein the light emitted by the light source comprises a visible light or an infrared light.

6. The head-mounted display of claim 4 , wherein the mobile device further comprises a communication unit configured to at least transmit data to a remote computing device, and wherein the processing unit is further configured to provide data associated with the reflected images of the first eye to the remote computing device through the communication unit.

7. The head-mounted display of claim 4 , wherein the mobile device further comprises a communication unit configured to at least transmit data to a remote computing device, and wherein the processing unit is further configured to perform operations comprising:
analyzing the reflected image of the first eye to determine a gaze direction of the first eye; and
providing data associated with the gaze direction to the remote computing device through the communication unit.

8. The head-mounted display of claim 4 , wherein the processing unit is further configured to perform operations comprising:
analyzing the reflected images of the first eye to determine a gaze direction of the first eye; and
based on the gaze direction, rendering a revised visual image displayable by the display unit.

9. The head-mounted display of claim 4 , wherein the processing unit is further configured to perform operations comprising:
analyzing the reflected images of the first eye to determine a gaze direction of the first eye; and
based on the gaze direction, performing foveated rendering.

10. The head-mounted display of claim 4 , wherein the processing unit is further configured to perform operations comprising:
analyzing the reflected images of the first eye to determine a behavior of the first eye which indicates a command for execution; and
executing the command.

11. The head-mounted display of claim 10 , wherein, in executing the command, the processing unit is configured to execute the command in a context of virtual reality (VR), and wherein the visual image comprises a VR image.

12. The head-mounted display of claim 10 , wherein the behavior of the first eye comprises fixation, saccades and blinking.

13. The head-mounted display of claim 4 , wherein the mobile device comprises a smartphone, a tablet computer, a phablet, or a portable computing device.

14. The head-mounted display of claim 4 , further comprising:
a wide angle lens disposed in front of the camera such that an angle of a field of view of the camera through the wide angle lens is at least enough to cover an observation target of interest.

15. The head-mounted display of claim 4 , further comprising:
a light guide coupled to the light source, the light guide configured to provide a point light in front of the at least one of the eyes of the user to produce a Purkinje image.

16. The head-mounted display of claim 15 , wherein the camera is configured to sense the Purkinje image, and wherein the processing unit is configured to determine a location of a pupil of the first eye by tracking a movement of the Purkinje image.

17. The head-mounted display of claim 1 , wherein the hot mirror is disposed between a display unit of the camera and the one or more lenses and also between the display unit and the reflective unit, and wherein the hot mirror is configured to reflect an infrared light while allowing a visible light to pass through.

18. The head-mounted display of claim 17 , wherein the camera is configured to sense visible and near infrared reflections.

19. A head-mounted display, comprising:
an eyewear piece comprising:
a holder that is wearable by a user and configured to hold the mobile device in front of eyes of the user;
one or more lenses disposed between the eyes of the user and the mobile device so that through the one or more lenses a display unit of the mobile device is viewable by the user; and
a reflective unit configured to reflect at least double images of at least a first eye of the eyes of the user,
wherein the reflective unit comprises a first mirror, a second mirror and a third mirror,
wherein the first mirror is disposed between the first eye of the user and the one or more lenses,
wherein the second mirror is disposed on the holder and facing toward the mobile device and away from the user,
wherein the third mirror is disposed between the display unit of the mobile device and the one or more lenses,
wherein a first image is reflected to a camera of the mobile device by the first mirror, and
wherein a second image is reflected to the camera by the third mirror to the second mirror, and further reflected by the second mirror to the camera.

20. The head-mounted display of claim 19 , wherein the third mirror comprises a hot mirror.

Assignees

• Mediatek Inc

Inventors

• Shiu Da-Shan
• Chang Kai-Mau
• Liu Tsu-Ming

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title