Head mounted display apparatus, method for operating the same, and computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-11960643-B2
Application number	US-202117427736-A
Country	US
Kind code	B2
Filing date	Jul 13, 2021
Priority date	Mar 15, 2021
Publication date	Apr 16, 2024
Grant date	Apr 16, 2024

Abstract

Provided are a head mounted display apparatus, an operating method thereof, and a storage medium. The head mounted display apparatus includes at least one display, at least one camera disposed on a back surface of the at least one display, and a controller, in which the controller is configured to obtain an image regarding both eyes of a user through the at least one camera, detect the both eyes of the user from the obtained image, measure an inter-pupillary distance (IPD) of the user based on the detected both eyes of the user, convert a picture based on the measured IPD, and display the converted picture on the at least one display.

First claim

The invention claimed is:

1. A head mounted display apparatus comprising:
at least one display to display a picture;
at least one camera under-display disposed behind the at least one display;
a controller;
a memory storing a plurality of instructions executable by the controller of the head mounted display apparatus, wherein execution of the instructions causes the head mounted display apparatus to performing a plurality of operations comprising:
obtaining, while the at least one display is mounted on a head of a user, an image of both eyes of the user through the at least one camera under-display;
identifying positions of the both eyes of the user from the obtained image;
determining an inter-pupillary distance (IPD) of the user based on the identified positions of the both eyes of the user and a gaze direction of the user from the obtained image;
converting the picture based on the determined IPD and the gaze direction to prevent VR motion sickness in the user; and
controlling the at least one display to display the converted picture based on the determined IPD and the gaze direction.

2. The head mounted display apparatus of claim 1 , wherein the plurality of operations further comprises:
detecting a gaze direction of the user from the obtained image; and
determining the IPD of the user based on the positions of each of the both eyes of the user in the obtained image and the detected gaze direction of the user in the obtained image.

3. The head mounted display apparatus of claim 2 , wherein the plurality of operations further comprises measuring the IPD of the user based on the positions of each of the both eyes of the user in the obtained image, the detected gaze direction of the user, and a position of the at least one camera.

4. The head mounted display apparatus of claim 1 , wherein the at least one display comprises a plurality of pixels and a plurality of micro lenses provided on a surface of the plurality of pixels facing the user.

5. The head mounted display apparatus of claim 1 , further comprising two optical lenses located at a first distance in the first surface of the at least one display and facing the user, and wherein the plurality of operations further comprises adjusting a distance between the two optical lenses based on the determined IPD.

6. The head mounted display apparatus of claim 1 , further comprising two displays, and wherein the plurality of operations further comprises adjusting a distance between the two displays based on the determined IPD.

7. The head mounted display apparatus of claim 1 , wherein the plurality of operation further comprises comparing the determined IPD with a preset IPD reference value; and converting the picture based on the comparison.

8. The head mounted display apparatus of claim 5 , wherein the plurality of operations further comprises comparing the measured IPD with a preset IPD reference value; and
adjust the distance between the two optical lenses based on the comparison.

9. The head mounted display apparatus of claim 8 , wherein the plurality of operations further comprises:
increasing the distance between the two optical lenses when the measured IPD is greater than the preset IPD reference value; and
decreasing the distance between the two optical lenses when the measured IPD is less than the preset IPD reference value.

10. The head mounted display apparatus of claim 6 , wherein the plurality of operations further comprises
comparing the measured IPD with a preset IPD reference value; and
adjusting the distance between the two displays based on the comparison.

11. A method of operating a head mounted display apparatus comprising at least one display to display a picture and at least one under-display camera, the method comprising:
obtaining, while the at least one display is mounted on a head of the user, an image of a left eye and a right eye of a user through the at least one under-display camera;
identifying positions of the both eyes of the user from the obtained image;
determining an inter-pupillary distance (IPD) of the user based on the identified positions of the both eyes of the user and a gaze direction of the user from the obtained image;
converting the picture based on the determined IPD to prevent VR motion sickness in the user; and
displaying the converted picture based on the determined IPD and the gaze direction on a first screen and a second screen of the at least one display.

12. The method of claim 11 , further comprising; detecting a gaze direction of the user from the obtained image, wherein the IPD of the user is determined based on the positions of each of the left eye and the right eye of the user in the obtained image and the detected gaze direction of the user in the obtained image.

13. The method of claim 11 , wherein the IPD of the user is measured based on the positions of each of the left eye and the right eye of the user in the obtained image, a detected gaze direction of the user, and a position of the at least one camera.

14. The method of claim 11 , wherein the at least one display comprises a plurality of pixels and a plurality of micro lenses provided on a surface of the plurality of pixels facing the user.

15. A computer-readable recording medium having stored therein a program of instructions, which when executed by a processor performs a method of operating a head mounted display apparatus comprising at least one display to display a picture and at least one under-display camera disposed, the method comprising:
obtaining, while the display is mounted on a head of a user, an image of both eyes of the user through the at least one under-display camera;
identifying positions of the both eyes of the user from the obtained image;
determining an inter-pupillary distance (IPD) of the user based on the identified positions of the both eyes of the user and a gaze direction of the user from the obtained image;
converting the picture based on the determined IPD to prevent VR motion sickness in the user; and
displaying the converted picture based on the determined IPD and the gaze direction.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Choi Jaesung
• Do Wonjoon
• Choi Woojhon
• Lee Kihuk

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30201

  Face · CPC title

• G02B27/0093 Primary

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B27/0172 Primary

  characterised by optical features · CPC title