Method and System for Calibrating a Wearable Heads-Up Display to Produce Aligned Virtual Images in an Eye Space

Patent metadata

Field	Value
Publication number	US-2020073119-A1
Application number	US-201916550500-A
Country	US
Kind code	A1
Filing date	Aug 26, 2019
Priority date	Aug 31, 2018
Publication date	Mar 5, 2020
Grant date	—

Abstract

A wearable heads-up display has stored therein two or more distortion models for use in generating distorted source images that when projected by the wearable heads-up display into a target eye space form virtual images are aligned in the target eye space. A method and system for determining the distortion models are disclosed.

First claim

1 . A method of calibrating a wearable heads-up display, the method comprising:
for each virtual image to be formed by the wearable heads-up display in a target eye space, determining a distortion model for use in distorting a source image corresponding to the virtual image, wherein determining the distortion model comprises:
projecting a set of test patterns by a projector of the wearable heads-up display in a select sequence;
capturing an image of each test pattern projected by the projector by a camera;
determining a first mapping that maps points in a projector space to points in a camera space from the captured images;
acquiring a second mapping that maps points in the camera space to points in the target eye space;
determining a third mapping that maps points in the target eye space to points in the projector space based on the first mapping and the second mapping; and
storing the third mapping as the distortion model for use in distorting the source image corresponding to the virtual image.

2 . The method of claim 1 , wherein determining a first mapping that maps points in a projector space to points in a camera space from the captured images comprises decoding the captured images to find correspondence between projector frame buffer pixels and camera pixels.

3 . The method of claim 2 , wherein decoding the captured images to find correspondence between projector frame buffer pixels and camera pixels comprises detecting Gray codes in the captured images.

4 . The method of claim 1 , wherein determining a third mapping that maps points in the target eye space to points in the projection space based on the first mapping and the second mapping comprises identifying at least one region of interest in the target eye space and determining the third mapping for the at least one region of interest in the target eye space.

5 . The method of claim 1 , wherein determining the distortion model for use in distorting a source image corresponding to the virtual image is performed at a first temperature to obtain a distortion model at the first temperature and at a second temperature to obtain the distortion model at the second temperature.

6 . The method of claim 1 , wherein projecting a set of test patterns by the projector of the wearable heads-up display in a select sequence comprises:
for each position in the select sequence, generating copies of the test pattern at the position and projecting the copies of the test pattern by the projector of the wearable heads-up display, wherein each of the copies corresponds to a unique combination of exit pupil and color channel of the wearable heads-up display.

7 . The method of claim 6 , wherein capturing an image of each test pattern projected by the projector by a camera comprises capturing an image having image portions corresponding to the copies of the test patterns by the camera.

8 . The method of claim 7 , wherein determining a first mapping that maps points in a projector space to points in a camera space from the captured images comprises determining the first mapping for each unique combination of color channel and exit pupil using the image portions of the captured image corresponding to the unique combination of color channel and exit pupil.

9 . The method of claim 1 , further comprising generating the set of test patterns prior to determining a distortion model for use in distorting a source image corresponding to the virtual image, wherein generating the set of test patterns comprises generating at least one pattern carrying codes.

10 . The method of claim 9 , wherein generating at least one pattern carrying codes comprises generating the at least one pattern with a Gray code of m bits, where m 1 .

11 . The method of claim 9 , wherein determining a first mapping that maps points in a projector space to points in a camera space from the captured images comprises decoding the at least one pattern carrying codes.

12 . The method of claim 9 , wherein generating the set of test patterns further comprises generating at least one additional pattern having features with known positions in a frame buffer of the projector.

13 . The method of claim 12 , wherein determining a first mapping that maps points in a projector space to points in a camera space from the captured images comprises decoding the at least one pattern carrying codes and the at least one additional pattern having features with known positions in the frame buffer of the projector.

14 . The method of claim 1 , wherein projecting a set of test patterns by a projector of the wearable heads-up display in a select sequence comprises transmitting the set of test patterns to a processor of the wearable heads-up display, wherein the processor of the wearable heads-up display renders the set of test patterns into a frame buffer of the projector in the select sequence.

15 . The method of claim 1 , wherein storing the third mapping as the distortion model for use in distorting the source image corresponding to the virtual image comprises storing the third mapping in a non-transitory processor-readable storage medium of the wearable heads-up display.

16 . The method of claim 1 , wherein capturing an image of each test pattern projected by the projector by a camera comprises positioning the camera relative to the wearable heads-up display to capture at least a portion of images projected by the wearable heads-up display.

17 . A wearable heads-up display calibration system, comprising:
a wearable heads-up display comprising a projector; a camera positioned and oriented to capture images projected by the projector; a calibration processor communicatively coupled to the wearable heads-up display and camera; and a non-transitory processor-readable storage medium communicatively coupled to the calibration processor, wherein the non-transitory processor-readable storage medium stores data and/or processor-executable instructions that, when executed by the calibration processor, cause the calibration processor to determine a distortion model that maps points from a target eye space to points in a projector space for each virtual image to be produced in the target eye space by the wearable heads-up display.

18 . The wearable heads-up display of claim 17 , wherein the wearable heads-up display further comprises a processor, and wherein the calibration processor is communicatively coupled to the processor of the wearable heads-up display.

19 . A method of forming a display UI on a wearable heads-up display, comprising:
generating N times K distorted source images, wherein N≥1 and K≥1 and N times K>1, by applying N times K different distortion models to K source images, wherein each of the K different distortion models maps points from a target eye space to points in a projector space corresponding to one of the K source images, wherein N is the number of separately addressable exit pupils of the display and K is the number of color channels of the display; and projecting at least two of the distorted source images from the projector space to the target eye space, wherein the at least two of the distorted source images form at least two virtual images in the target eye space that are aligned in the target eye space.

20 . The method of claim 19 , wherein generating N times K distorted source images comprises applying the N times K different distortion models to the K source images when rendering the K source images into a projector frame buffer of the wearable heads-up display.

21 . Th

Assignees

• North Inc

Inventors

• Urquhart Daniel Thomson
• Balzerson Daniel
• Kintel Marius

Classifications

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/0172 Primary

  characterised by optical features · CPC title

• G02B2027/011

  comprising device for correcting geometrical aberrations, distortion · CPC title

• G02B2027/0181

  Adaptation to the pilot/driver · CPC title

• H04N9/3194

  including sensor feedback · CPC title