Positioning cameras in a head mounted display to capture images of portions of a face of a user

Patent metadata

Field	Value
Publication number	US-10495882-B1
Application number	US-201815997655-A
Country	US
Kind code	B1
Filing date	Jun 4, 2018
Priority date	Jun 4, 2018
Publication date	Dec 3, 2019
Grant date	Dec 3, 2019

Abstract

A virtual reality (VR) or augmented reality (AR) head mounted display (HMD) includes multiple image capture devices positioned within and on the HMD to capture portions of a face of a user wearing the HMD. Multiple image capture devices are included within the HMD to capture different portions of the face of the user within the HMD, and one or more other image capture devices are positioned to capture portions of the face of the user external to the HMD. Captured images from various image capture devices may be communicated to a console or a controller that generates a graphical representation of the user's face based on the captured images.

First claim

What is claimed is:

1. A head mounted display (HMD) comprising:
a rigid body having a front side, a left side, a right side, a top side, and a bottom side and including a display element configured to display content to a user wearing the HMD and an optics block configured to direct light from the display element to an exit pupil and an additional exit pupil of the HMD;
a left image capture device coupled to an interior surface of the left side of the front rigid body and proximate to the bottom side of the rigid body, the left image capture device configured to capture images of a portion of the user's face enclosed by the rigid body;
an right image capture device coupled to an interior surface of the right side of the front rigid body and proximate to the bottom side of the rigid body, the right image capture device configured to capture images of an additional portion of the user's face enclosed by the rigid body; and
a central image capture device positioned within the rigid body between the exit pupil and the additional exit pupil, the central image capture device configured to capture images of a central portion of the user's face enclosed by the rigid body.

2. The HMD of claim 1 , further comprising:
a left illumination source positioned proximate to the left image capture device and configured to emit light illuminating the portion of the user's face.

3. The HMD of claim 2 , wherein the left illumination source is configured to emit light while the left image capture device captures an image of the portion of the user's face and to not emit light when the left image capture device is not capturing the image of the portion of the user's face.

4. The HMD of claim 2 , further comprising:
a right illumination source positioned proximate to the right image capture device and configured to emit light illuminating the additional portion of the user's face.

5. The HMD of claim 4 , wherein the right illumination source is configured to emit light while the right image capture device captures an image of the additional portion of the user's face and to not emit light when the right image capture device is not capturing the image of the additional portion of the user's face.

6. The HMD of claim 4 , further comprising:
a central illumination source positioned proximate to the central image capture device and configured to emit light illuminating the central portion of the user's face.

7. The HMD of claim 6 , wherein the central illumination source is configured to emit light while the central image capture device captures an image of the central portion of the user's face and to not emit light when the central image capture device is not capturing the image of the central portion of the user's face.

8. The HMD of claim 1 , wherein the left image capture device, the right image capture device, and the central image capture device have complementary fields of view.

9. The HMD of claim 1 , further comprising:
an external image capture device coupled to the bottom side of the front rigid body and configured to capture images of a portion of the user's face external to the bottom surface of the front rigid body.

10. The HMD of claim 9 , further comprising:
an external illumination source positioned proximate to the external image capture device and configured to emit light illuminating the portion of the user's face external to the bottom surface of the front rigid body.

11. A system comprising:
a head mounted display (HMD) comprising:
a rigid body having a front side, a left side, a right side, a top side, and a bottom side and including a display element configured to display content to a user wearing the HMD and an optics block configured to direct light from the display element to an exit pupil and an additional exit pupil of the HMD;
a left image capture device coupled to an interior surface of the left side of the front rigid body and proximate to the bottom side of the rigid body, the left image capture device configured to capture images of a portion of the user's face enclosed by the rigid body;
an right image capture device coupled to an interior surface of the right side of the front rigid body and proximate to the bottom side of the rigid body, the right image capture device configured to capture images of an additional portion of the user's face enclosed by the rigid body;
a central image capture device positioned within the rigid body between the exit pupil and the additional exit pupil, the central image capture device configured to capture images of a central portion of the user's face enclosed by the rigid body; and
a controller coupled to the left image capture device, to the right image capture device, and to the central image capture device, the controller configured to generate a graphical representation of the user's face based on the images of the portion of the user's face, the images of the additional portion of the user's face, and the images of the central portion of the user's face; and

a console coupled to the HMD, the console configured to provide content to the HMD for presentation by an electronic display included in the HMD and to receive the graphical representation of the user's face from the controller.

12. The system of claim 11 , wherein the HMD further comprises:
a left illumination source positioned proximate to the left image capture device and configured to emit light illuminating the portion of the user's face.

13. The system of claim 12 , wherein the left illumination source is configured to emit light while the left image capture device captures an image of the portion of the user's face and to not emit light when the left image capture device is not capturing the image of the portion of the user's face.

14. The system of claim 12 , wherein the HMD further comprises:
a right illumination source positioned proximate to the right image capture device and configured to emit light illuminating the additional portion of the user's face.

15. The system of claim 14 , wherein the right illumination source is configured to emit light while the right image capture device captures an image of the additional portion of the user's face and to not emit light when the right image capture device is not capturing the image of the additional portion of the user's face.

16. The system of claim 14 , wherein the HMD further comprises:
a central illumination source positioned proximate to the central image capture device and configured to emit light illuminating the central portion of the user's face.

17. The system of claim 16 , wherein the central illumination source is configured to emit light while the central image capture device captures an image of the central portion of the user's face and to not emit light when the central image capture device is not capturing the image of the central portion of the user's face.

18. The system of claim 11 , wherein the left image capture device, the right image capture device, and the central image capture device have complementary fields of view.

19. The system of claim 11 , wherein the HMD further comprises:
an external image capture device coupled to the bottom side of the front rigid body and configured to capture images of a portion of the user's face external to the bottom surface of the front rigid body.

20. The system of claim 19 , wherein the HMD further comprises:
an external illumination source positioned proximate to the external image capture device and configured to emit light illuminating the portion of the user's face ex

Assignees

• Facebook Tech Llc

Inventors

• Badino Hernan
• Sheikh Yaser
• Hypes Alexander Trenor
• Wang Dawei
• Shahmohammadi Mohsen
• Perdoch Michal
• Saragih Jason
• Wei Shih-En

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B27/0172 Primary

  characterised by optical features · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06T7/75

  involving models · CPC title