NTE display systems and methods with optical trackers

Patent metadata

Field	Value
Publication number	US-10466774-B2
Application number	US-201615358383-A
Country	US
Kind code	B2
Filing date	Nov 22, 2016
Priority date	Nov 22, 2016
Publication date	Nov 5, 2019
Grant date	Nov 5, 2019

Abstract

An optical tracker system is provided for tracking a location and orientation of a first object. The system includes a prediction module configured to determine a predicted location and orientation of the first object as a spatial prediction; a display module configured to generate display commands for a fiducial symbol based on the spatial prediction; a display unit configured to display the fiducial symbol based on the display commands; a camera unit configured to capture an image of the fiducial symbol displayed on the display device as a captured appearance; and a position determination module coupled to the camera unit and configured to receive the image from camera unit. The position determination module is configured to identify the fiducial symbol and determine the location and the orientation of the first object based on the captured appearance of the fiducial symbol.

First claim

What is claimed is:

1. An optical tracker system for tracking a location and orientation of a first object within an operational environment, comprising:
a processor programmed to implement a prediction module that determines a predicted location and orientation of the first object as a spatial prediction;
a display module implemented by the processor, the display module coupled to receive the spatial prediction, the display module configured to generate display commands for a fiducial symbol based on the spatial prediction;
a tracker display unit arranged within the operational environment and separated from the first object, the tracker display unit coupled to the prediction module and configured to receive the display commands, the tracker display unit configured to display the fiducial symbol based on the display commands;
a camera unit mounted on the first object and configured to capture an image of the fiducial symbol displayed on the tracker display unit as a captured appearance of the displayed fiducial symbol; and
a position determination module implemented by the processor, the position determination module coupled to the camera unit and configured to receive the image of the fiducial symbol from the camera unit, the position determination module configured to identify the fiducial symbol and determine the location and the orientation of the first object based on the captured appearance of the displayed fiducial symbol.

2. The optical tracker system of claim 1 , wherein the tracker display unit is an active matrix display device.

3. The optical tracker system of claim 2 , wherein the tracker display unit is formed on a single substrate.

4. The optical tracker system of claim 1 , wherein the display module is configured to display the fiducial symbol on the tracker display unit with a nominal appearance from the perspective of the spatial prediction.

5. The optical tracker system of claim 1 , wherein the position determination module is configured to compare the captured appearance of the fiducial symbol to the nominal appearance to determine the location and orientation of the first object.

6. The optical tracker system of claim 1 , wherein the position determination module is configured to determination the location and orientation of the first object with six degrees-of-freedom (DOF).

7. The optical tracker system of claim 1 , further comprising a structure configured to be disposed on a head of a subject as the first object.

8. The optical tracker system of claim 1 , further comprising:
a structure configured to be disposed on a head of a subject as the first object; and
a structure display unit mounted on and moveable with the structure.

9. The optical tracker system of claim 1 , wherein the display module is configured to display the fiducial symbol on the tracker display unit as a grid of intersecting lines with a non-symmetrical feature.

10. The optical tracker system of claim 1 , wherein the display module is configured to display the fiducial symbol on the tracker display unit as a nested pair of rotated fiducial symbols.

11. The optical tracker system of claim 1 , further comprising an optical modifier interposed between the tracker display unit and the camera unit configured to alter an appearance of the fiducial symbol for the image of the camera unit.

12. The optical tracker system of claim 11 , wherein the optical modifier includes at least one of a refractive element, an aspheric lens, multiple parallel reflective surface elements, or a plurality of specularly reflective mirrors.

13. A method for tracking a location and orientation of a first object with an optical tracker system, the method comprising:
determining a predicted location and orientation of the first object within an operational environment as a spatial prediction;
generating display commands for a fiducial symbol based on the spatial prediction;
displaying the fiducial symbol based on the display commands on a tracker display device, wherein the tracker display device disposed within the operational environment and separated from the first object;
capturing, from a location and orientation on the first object, an image of the fiducial symbol displayed on the display device as a captured appearance;
identifying the fiducial symbol in the image; and
determining the location and the orientation of the first object based on the captured appearance of the displayed fiducial symbol.

14. The method of claim 13 , wherein the tracker display unit is an active matrix display device.

15. The method of claim 13 ,
wherein the displaying step includes displaying the fiducial symbol on the tracker display unit with a nominal appearance from the perspective of the spatial prediction, and
wherein the determining step includes comparing the captured appearance of the fiducial symbol to the nominal appearance to determine the location and orientation of the first object.

16. The method of claim 13 , wherein the determining step includes determining the location and orientation of the first object with six degrees-of-freedom (DOF).

17. The method of claim 13 , further comprising positioning the first object on a structure configured to be disposed on a head of a subject, wherein the capturing step is performed by a camera unit mounted on and moving with the structure.

18. The method of claim 13 , wherein the displaying step includes displaying the fiducial symbol on the tracker display unit as a nested pair of rotated fiducial symbols.

19. A near-to-eye display system, comprising:
an optical tracker system for tracking a location and orientation of a first object, comprising
a first display unit comprising an active matrix display device, the first display unit positioned at a fixed location and disposed separate from the first object and configured to display a fiducial symbol based on first display commands;
a camera unit mounted on the first object and configured to capture an image of the fiducial symbol displayed on the first display unit as a captured appearance of the displayed fiducial symbol; and
a processor implementing a position determination module and coupled to the camera unit, the processor implemented position determination module coupled to receive the image from the camera unit and configured to (i) identify the fiducial symbol and (ii) determine the location and the orientation of the first object based on the captured appearance of the displayed fiducial symbol;

a drive unit coupled to the optical tracker system and configured to generate second display commands based on the location and the orientation of the first object; and
a second display unit coupled to the drive unit and configured to render symbology based on the second display commands for viewing by a user relative to a surrounding environment whereby the user may view the rendered symbology in appropriate positions.

20. The near-to-eye display system of claim 19 , wherein the processor further implements:
a prediction module that is configured to determine a predicted location and orientation of the first object as a spatial prediction; and
a display module that receives the spatial prediction and generates the first display commands for the fiducial symbol based on the spatial prediction.

Assignees

• Honeywell Int Inc

Inventors

• Larson Brent D

Classifications

• G06T7/74

  involving reference images or patches · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B27/0093 Primary

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title