Optical see through (OST) head mounted display (HMD) system and method for precise alignment of virtual objects with outwardly viewed objects

Patent metadata

Field	Value
Publication number	US-12079385-B2
Application number	US-202118557618-A
Country	US
Kind code	B2
Filing date	Apr 27, 2021
Priority date	Apr 27, 2021
Publication date	Sep 3, 2024
Grant date	Sep 3, 2024

Abstract

A method for irradiating an image in an optical see-through (OST) head mounted display (HMD) for viewing through, the OST HMD by a user's eye, an object having at least one of known orientation and position and orientation (O/P&O), associated with a first reference frame, the method comprising: generating and irradiating said image for appearing to said user superimposed in an aligned manner to said object, according to predetermined information, eyeball feature position data, and said O/P&O; said predetermined information relates correction data with a plurality of different respective position data values of at least one eyeball feature position of said eye; said predetermined information further includes display corrections of said electro-optical display module with respect to said position data values of said at least one eyeball feature position, with respect to a second reference frame; and said O/P&O is between said second reference and said first reference frame.

First claim

The invention claimed is:

1. An optical see-through (OST) head mounted display (HMD) system, for viewing an image aligned with an object associated with a first reference frame, the OST HMD system comprising:
an electro-optical display module including:
at least one partially reflective partially transmissive optical element for at least one of viewing therethrough said object, and viewing said image; and

at least one electro-optical projection module configured to irradiate said image for viewing by at least one eye of a user who wears said OST HMD, said electro-optical display module being associated with a second reference frame; and
a processor configured to be coupled with said electro-optical display module, said processor configured to generate said image, according to predetermined information, eyeball feature position data, and at least one of an orientation, and a position and orientation, so that said image appears to said user in an aligned manner with respect to said object;
wherein said eyeball feature position data is associated with at least one eyeball feature position of said at least one eye, with respect to said second reference frame;
wherein said at least one of orientation, and position and orientation is of said second reference frame with respect to said first reference frame; and
wherein said predetermined information relates correction data of said OST HMD to a plurality of different respective position data values of said at least one eyeball feature position;
wherein said predetermined information further includes display corrections of said electro-optical display module with respect to said position data values of said at least one eyeball feature position,
wherein said predetermined information further includes see-through corrections of said at least one partially reflective partially transmissive optical element,
wherein said predetermined information is attained via a calibration procedure with respect to said plurality of different respective position data, using a calibration system,
wherein said object is either one of at least partially appearing outwardly and completely hidden to said user through said at least one partially reflective partially transmissive optical element when said image is irradiated by said at least one optical projection module.

2. The system according to claim 1 , further comprising a position determination module, configured to determine said at least one eyeball feature position of said at least one eye with respect to said second reference frame, and to generate corresponding said eyeball feature position data.

3. The system according to claim 2 , wherein said electro-optical display module and said position determination module are mechanically coupled.

4. The system according to claim 2 , wherein said electro-optical display module and said position determination module are configured as an on-axis optical arrangement.

5. The system according to claim 1 , further comprising a tracking system configured to determine said at least one of an orientation, and a position and orientation, of said second reference frame with respect to said first reference frame.

6. The system according to claim 1 , further including a memory device for storing said predetermined information, wherein said processor is configured to read said predetermined information from said memory device.

7. The system according to claim 1 , wherein in case said at least one eyeball feature position of said at least one eye does not precisely correspond with one of said plurality of different respective position data values of said predetermined information, said processor is configured to derive from said correction data, closest-matching correction data associated with at least one closest matching position.

8. The system according to claim 1 , wherein said OST HMD is used for surgical applications where said hidden object is an anatomical feature.

9. A method for irradiating an image in an optical see-through (OST) head mounted display (HMD) for viewing through the OST HMD an object having at least one of a known orientation and a known position and orientation, with respect to a first reference frame, by at least one eye of a user, the method comprising:
an initial calibration procedure for acquiring predetermined information with respect to a plurality of different respective eyeball feature position data;
generating and irradiating said image for viewing by said user, such that said image appears to said user superimposed in an aligned manner with respect to said object that is either one of at least partially appearing outwardly and completely hidden to said user, according to said predetermined information, said eyeball feature position data, and at least one of an orientation, and a position and orientation;
wherein said predetermined information relates correction data of said OST HMD with a plurality of different respective position data values of at least one eyeball feature position of said at least one eye;
wherein said predetermined information further includes display corrections of said electro-optical display module with respect to said position data values of said at least one eyeball feature position,
wherein said predetermined information further includes see-through corrections of said at least one partially reflective partially transmissive optical element,
wherein said eyeball feature position data is associated with at least one eyeball feature position of said at least one eye, with respect to a second reference frame; and
wherein said at least one of orientation and position and orientation is of said second reference frame with respect to said first reference frame.

10. The method according to claim 9 , further comprising generating said eyeball feature position data, by determining corresponding said at least one eyeball feature position.

11. The method according to claim 10 , further comprising deriving closest-matching correction data in case determined said at least one eyeball feature position of said at least one eye does not precisely correspond with one of said plurality of different respective position data values of said predetermined information.

12. The method according to claim 10 , wherein said determining said at least one eyeball feature position and generating and irradiating said image is performed along a common optical axis.

13. The method according to claim 10 , wherein said determining said at least one eyeball feature position is performed separately from at least two different spatial positions.

14. The method according to claim 9 , further comprising determining said at least one of an orientation, and a position and orientation, of said second reference frame with respect to said first reference frame.

15. The method according to claim 9 , further comprising storing said predetermined information.

Assignees

• Elbit Systems Ltd

Inventors

• Ben-Yishai Rani
• Benesh Gil

Classifications

• G02B27/0172

  characterised by optical features · CPC title

• H04N13/327

  Calibration thereof · CPC title

• H04N13/344

  with head-mounted left-right displays · CPC title

• A61B2034/258

  providing specific settings for specific users · CPC title

• A61B2017/00216

  with eye tracking or head position tracking control · CPC title