Hand And Body Tracking With Mobile Device-Based Virtual Reality Head-Mounted Display

Patent metadata

Field	Value
Publication number	US-2016378176-A1
Application number	US-201514748231-A
Country	US
Kind code	A1
Filing date	Jun 24, 2015
Priority date	Jun 24, 2015
Publication date	Dec 29, 2016
Grant date	—

Abstract

A head-mounted display (HMD) may include a mobile device that includes a display unit, at least one sensing unit and a processing unit. The at least one sensing unit may be configured to detect a presence of an object. The processing unit may be configured to receive data associated with the detecting from the at least one sensing unit, and determine one or more of a position, an orientation and a motion of the object based at least in part on the received data. The HMD may also include an eyewear piece that includes a holder and a field of view (FOV) enhancement unit. The holder may be wearable by a user on a forehead thereof to hold the mobile device in front of eyes of the user. The FOV enhancement unit may be configured to enlarge or redirect a FOV of the at least one sensing unit.

First claim

What is claimed is:

1 . A head-mounted display, comprising:
an eyewear piece comprising:
a holder that is wearable by a user on a forehead thereof, the holder configured to hold a mobile device in front of eyes of the user; and
a field of view (FOV) enhancement unit configured to enlarge or redirect a FOV of one or more sensing units of the mobile device when the mobile device is held by the holder.

2 . The head-mounted display of claim 1 , wherein the FOV enhancement unit comprises a reflective element.

3 . The head-mounted display of claim 2 , wherein the reflective element comprises a mirror or an optical prism.

4 . The head-mounted display of claim 1 , wherein the FOV enhancement unit comprises a wide angle lens.

5 . The head-mounted display of claim 1 , wherein the FOV enhancement unit is configured to redirect the FOV of the at least one sensing unit toward a body part of interest of the user.

6 . The head-mounted display of claim 5 , wherein the body part of interest comprises at least a hand of the user.

7 . The head-mounted display of claim 1 , wherein the holder comprises a pair of goggles configured to seal off a space between the eyewear piece and a face of the user to prevent an ambient light from entering the space.

8 . The head-mounted display of claim 1 , wherein the eyewear piece further comprises a motion sensor configured to sense a motion of the eyewear piece and output a motion sensing signal indicative of the sensed motion.

9 . The head-mounted display of claim 1 , further comprising:
a mobile device having a first primary side and a second primary side opposite the first primary side, the mobile device comprising:
a display unit on the first primary side;
at least one sensing unit on the second primary side, the at least one sensing unit configured to detect a presence of an object; and
a processing unit configured to control operations of the display unit and the at least one sensing unit, the processing unit also configured to receive data associated with the detecting from the at least one sensing unit, the processing unit further configured to determine one or more of a position, an orientation and a motion of the object based at least in part on the received data.

10 . The head-mounted display of claim 9 , wherein the mobile device comprises a smartphone, a tablet computer, a phablet, or a portable computing device.

11 . The head-mounted display of claim 9 , wherein the at least one sensing unit comprises a camera, dual cameras, or a depth camera.

12 . The head-mounted display of claim 9 , wherein the at least one sensing unit comprises an ultrasound sensor.

13 . The head-mounted display of claim 9 , wherein the at least one sensing unit comprises a camera and an ultrasound sensor.

14 . The head-mounted display of claim 9 , wherein the at least one sensing unit comprises a camera, and wherein the processing unit is configured to perform one or more operations comprising increasing a frame rate of the camera, lowering a resolution of the camera, adopting 2×2 or 4×4 pixel binning or partial pixel sub-sampling, or deactivating an auto-focus function of the camera.

15 . The head-mounted display of claim 9 , wherein the at least one sensing unit comprises a motion sensor configured to sense a motion of the mobile device and output a motion sensing signal indicative of the sensed motion, and wherein the processing unit is configured to receive the motion sensing signal from the motion sensor and compensate for the sensed motion in a virtual reality (VR) application.

16 . The head-mounted display of claim 9 , wherein the eyewear piece further comprises a motion sensor configured to sense a motion of the eyewear piece and output a motion sensing signal indicative of the sensed motion, and wherein the processing unit is configured to receive the motion sensing signal from the motion sensor and compensate for the sensed motion in a virtual reality (VR) application.

17 . The head-mounted display of claim 9 , wherein the mobile device further comprises a wireless communication unit configured to at least wirelessly receive a signal from a wearable computing device worn by the user.

18 . The head-mounted display of claim 17 , wherein the processing unit is also configured to determine one or more of the position, the orientation and the motion of the object based on the received data and the received signal.

19 . The head-mounted display of claim 9 , wherein the mobile device further comprises an image signal processor (ISP) configured to provide a first mode and a second mode, the first mode optimized for general photography, the second mode optimized for continuous tracking, analysis and decoding of information related to tracking of the object.

20 . The head-mounted display of claim 9 , wherein the FOV enhancement unit comprises a wide angle lens, and wherein the at least one sensor comprises a camera, and wherein the wide angle lens is disposed in front of the camera such that an angle of a FOV of the camera through the wide angle lens is at least enough to cover an observation target of interest.

21 . The head-mounted display of claim 9 , wherein the processing unit is also configured to render a visual image displayable by the display unit in a context of virtual reality (VR).

22 . The head-mounted display of claim 21 , wherein the visual image corresponds to the detected object.

23 . A head-mounted display, comprising:
a mobile device having a first primary side and a second primary side opposite the first primary side, the mobile device comprising:
a display unit on the first primary side;
at least one sensing unit on the second primary side, the at least one sensing unit configured to detect a presence of an object, the at least one sensing unit comprising one or two cameras, a depth camera, an ultrasound sensor, or a combination thereof; and
a processing unit configured to control operations of the display unit and the at least one sensing unit, the processing unit also configured to receive data associated with the detecting from the at least one sensing unit, the processing unit further configured to determine one or more of a position, an orientation and a motion of the object based at least in part on the received data; and
an eyewear piece comprising:
a holder that is wearable by a user on a forehead thereof, the holder configured to hold the mobile device in front of eyes of the user; and
a field of view (FOV) enhancement unit configured to enlarge or redirect a FOV of the at least one sensing unit by redirecting the FOV of the at least one sensing unit toward a body part of interest of the user, the FOV enhancement unit comprising a mirror, a wide angle lens or an optical prism.

24 . The head-mounted display of claim 23 , wherein the at least one sensing unit comprises a motion sensor configured to sense a motion of the mobile device and output a motion sensing signal indicative of the sensed motion, and wherein the processing unit is configured to receive the motion sensing signal from the motion sensor and compensate for the sensed motion in a virtual reality (VR) application.

25 . The head-mounted display of claim 23 , wherein the eyewear piece further comprises a motion sensor configured to sense a motion of the eyewear piece and output a motion sensing signal indicative of the sensed motion, and wherein the processing uni

Assignees

• Mediatek Inc

Inventors

• Shiu Da-Shan
• Chang Kai-Mau

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title

• H04M1/05

  specially adapted for use on head, throat or breast · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/017

  Head mounted · CPC title