Compensation for deformation in head mounted display systems

Patent metadata

Field	Value
Publication number	US-12175644-B2
Application number	US-202318454912-A
Country	US
Kind code	B2
Filing date	Aug 24, 2023
Priority date	Jan 15, 2020
Publication date	Dec 24, 2024
Grant date	Dec 24, 2024

Abstract

The systems and methods described can include approaches to calibrate head-mounted displays for improved viewing experiences. Some methods include receiving data of a first target image associated with an undeformed state of a first eyepiece of a head-mounted display device; receiving data of a first captured image associated with deformed state of the first eyepiece of the head-mounted display device; determining a first transformation that maps the first captured image to the image; and applying the first transformation to a subsequent image for viewing on the first eyepiece of the head-mounted display device.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving data of a first target image associated with an undeformed state of a first eyepiece of a head-mounted display device, the first eyepiece comprising a first projector that is optically coupled to the first eyepiece;
transmitting the data of the first target image to the first projector;
receiving data of a first captured image that is associated with a deformed state of the first eyepiece, wherein the first captured image is received by a first imaging sensor that is optically coupled to the first projector; and
determining a first transformation, wherein the first transformation maps the first captured image to the first target image.

2. The computer-implemented method of claim 1 , further comprising:
transmitting the transformed subsequent image to the first projector optically coupled to the first eyepiece.

3. The computer-implemented method of claim 1 , further comprising:
capturing the data of the first captured image by the first imaging sensor.

4. The computer-implemented method of claim 1 , further comprising:
receiving a temperature measurement of the first imaging sensor; and
accounting for the received temperature measurement in the first transformation.

5. The computer-implemented method of claim 1 , further comprising:
receiving data from one or more sensors representing a state of a real-world environment external to the head-mounted display device, wherein the first transformation aligns features of the real-world environment with corresponding features from the first target image.

6. The computer-implemented method of claim 1 , wherein the first transformation aligns each individual color channel of the first captured image with each individual color channel of the first target image.

7. The computer-implemented method of claim 1 , further comprising:
receiving a trigger signal, wherein determining the first transformation is responsive to receiving the trigger signal.

8. The computer-implemented method of claim 1 , wherein the first eyepiece of the head-mounted display device is optically transparent and wherein the first transformation accounts for a relative position of the first imaging sensor to the first eyepiece.

9. The computer-implemented method of claim 1 , further comprising:
receiving data of a second target image associated with an undeformed state of a second eyepiece of the head-mounted display device, the second eyepiece comprising a second projector that is optically coupled to the second eyepiece;
receiving data of a second captured image associated with deformed state of the second eyepiece, wherein the second captured image is received by a second imaging sensor optically coupled to a second projector of the second eyepiece, the second imaging sensor being part of the head-mounted display device; and
determining a second transformation, wherein the second transformation maps the second captured image to the second target image.

10. The computer-implemented method of claim 9 , wherein the first transformation is dependent on the second transformation and wherein the second transformation aligns subsequent images for viewing on the second eyepiece with subsequent images for viewing on the first eyepiece.

11. The computer-implemented method of claim 9 , wherein the first imaging sensor is rigidly connected to the second imaging sensor and wherein the first eyepiece and the second eyepiece are deformable relative to the first imaging sensor and the second imaging sensor.

12. The computer-implemented method of claim 9 , wherein the second transformation accounts for a relative position of the second imaging sensor relative to the second eyepiece.

13. The computer-implemented method of claim 9 , further comprising:
receiving a temperature measurement of the second imaging sensor; and
accounting for the received temperature measurement in the second transformation.

14. A head-mounted display device comprising:
a wearable frame;
a first eyepiece elastically mounted to the wearable frame, the first eyepiece comprising a first projector rigidly and optically coupled to the first eyepiece, the first projector being configured to emit light to the first eyepiece;
a first imaging sensor elastically mounted to the wearable frame, optically coupled to the first projector, and configured to capture emitted light from the first eyepiece; and
a processor communicatively coupled to the first projector and the first imaging sensor, the processor configured to:
receive data of a first target image associated with an undeformed state of the first eyepiece;
transmit data of the first target image to the first projector;
receive data from the first imaging sensor of a first captured image representing a deformed state of the first eyepiece; and
determine a first transformation, wherein the first transformation maps the first captured image to the first target image.

15. The head-mounted display device of claim 14 , further comprising:
one or more sensors associated mounted to the wearable frame and configured to receive data regarding a state of a real-world environment external to the head-mounted display device, wherein the first transformation aligns features of the real-world environment with corresponding features from the first target image.

16. The head-mounted display device of claim 14 , further comprising:
a temperature sensor configured to measure a temperature of the first imaging sensor, and wherein the first transformation is based, at least in part, on the measured temperature of the first imaging sensor; and
a trigger sensor configured to transmit a signal to the processor for initiating the process performed by the processor, wherein the first eyepiece is optically transparent.

17. The head-mounted display device of claim 14 , further comprising:
a second eyepiece elastically mounted to the wearable frame, the second eyepiece comprising a second projector rigidly and optically coupled to the second eyepiece, the second projector being configured to emit light to the second eyepiece;
a second imaging sensor elastically mounted to the wearable frame and configured to capture emitted light from the second eyepiece; and
wherein the processor is further configured to:
transmit data of a second target image to the second projector;
receive data from the second imaging sensor of a second captured image representing a deformed state of the second eyepiece; and
determine a second transformation, wherein the second transformation maps the second captured image to the second target image.

18. The head-mounted display device of claim 17 , wherein the second transformation aligns features of a real-world environment with corresponding features from the second target image and wherein the second transformation is dependent on the first transformation.

19. The head-mounted display device of claim 17 , wherein the second transformation accounts for a relative position of the second imaging sensor relative to the second eyepiece.

20. The head-mounted display device of claim 17 , further comprising:
receiving a temperature measurement of the second imaging sensor; and
accounting for the received temperature measurement in the second transformation.

Assignees

• Magic Leap Inc

Inventors

• Edwin Lionel Ernest
• Miller Samuel A
• Grossmann Etienne Gregoire
• Clark Brian Christopher
• Johnson Michael Robert
• Zhao Wenyi
• Shah Nukul Sanjay
• Huang Po-Kang

Classifications

• G06T3/14

  Transformations for image registration, e.g. adjusting or mapping for alignment of images · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06T2207/10024

  Color image · CPC title