Systems and methods for projecting images onto an object

Patent metadata

Field	Value
Publication number	US-9520072-B2
Application number	US-201213624371-A
Country	US
Kind code	B2
Filing date	Sep 21, 2012
Priority date	Sep 21, 2011
Publication date	Dec 13, 2016
Grant date	Dec 13, 2016

Abstract

In one embodiment, a method for projecting images on a subject includes determining a pose and position of the subject, adjusting a three-dimensional model of an anatomical structure of the subject to match the determined pose and position, and projecting an image of the anatomical structure onto the subject in registration with the actual anatomical structure of the subject to illustrate the location of the structure on or within the subject.

First claim

The invention claimed is:

1. A method for providing education to a student as to an internal anatomical structure, the method comprising:
continually tracking over time a living subject with a motion tracking device external to the subject;
continually estimating the subject's real-time skeletal pose and position over the time based on data obtained from the motion tracking device and continually determining the real-time pose and position of the subject's internal anatomical structure over the time based on the estimated skeletal pose and position;
continually adjusting a three-dimensional model of the internal anatomical structure to match the determined pose and position; and
continually projecting images of the internal anatomical structure onto the subject in registration with the actual internal anatomical structure of the subject to illustrate the location of the structure within the subject to a student, wherein the images of the internal anatomical structure remain in registration with the subject's actual internal anatomical structure as the subject moves because of the continual tracking, adjusting, and projecting.

2. The method of claim 1 , wherein the subject is a human subject and the anatomical structure is an internal anatomical structure.

3. The method of claim 2 , wherein the internal anatomical structure is an internal organ.

4. The method of claim 2 , wherein the internal anatomical structure is a bone.

5. The method of claim 1 , wherein continually projecting images comprises continually updating the images projected onto the subject so that the anatomical structure maintains its registration with the actual anatomical structure as the subject moves.

6. The method of claim 1 , further comprising detecting a gesture made by the subject and performing in response an action associated with the gesture.

7. The method of claim 6 , wherein the gesture comprises the subject pointing to a particular anatomical structure in the projected image.

8. The method of claim 6 , wherein the action comprises projecting onto the subject a particular type of anatomical structure associated with the gesture.

9. The method of claim 6 , further comprising quizzing the subject as to the anatomical structure.

10. A non-transitory computer-readable medium that stores an image projection system configured to provide education to a student as to an internal anatomical structure, comprising:
logic configured to continually estimate a skeletal pose and position of a living subject over time from data collected by continually tracking the subject with a motion tracking device external to the subject and to continually determine the real-time pose and position of the subject's internal anatomical structure over the time based on the estimated skeletal pose and position;
logic configured to continually adjust a three-dimensional model of the internal anatomical structure of the subject to match the determined pose and position; and
logic configured to continually generate images of the internal anatomical structure for projection onto the subject in registration with the actual internal anatomical structure of the subject to illustrate the location of the structure within the subject to a student, wherein the images of the internal anatomical structure remain in registration with the subject's actual internal anatomical structure as the subject moves because of the continual tracking, adjusting, and generating.

11. The non-transitory computer-readable medium of claim 10 , wherein the logic configured to estimate a skeletal pose and position comprises logic configured to estimate the skeletal pose and position from images captured with a motion tracking device configured to track movement of the subject.

12. The non-transitory computer-readable medium of claim 10 , wherein the logic configured to continually generate images is configured to continually update the images to be projected onto the subject so that the anatomical structure maintains its registration with the actual anatomical structure as the subject moves.

13. The non-transitory computer-readable medium of claim 10 , further comprising logic configured to detect gestures made by the subject and to perform actions associated with the gestures.

14. The non-transitory computer-readable medium of claim 10 , further comprising logic configured to quiz the subject about the anatomical structures.

15. A system for providing education to a student as to an internal anatomical structure, the system comprising:
an external motion tracking device configured to continually track a living subject over time;
a projector configured to continually project images onto the subject; and
a computing device configured to (i) continually receive data from the motion tracking device, continually estimate a skeletal pose and position of the subject from the data, and continually determine the real-time pose and position of the subject's internal anatomical structure over the time based on the estimated skeletal pose and position, (ii) continually adjust a three-dimensional model of the internal anatomical structure to match the determined pose and position, and (iii) continually provide updated images of the internal anatomical structure to the projector for projection onto the subject in registration with the actual internal anatomical structure of the subject to illustrate the location of the structure within the subject to a student, wherein the images of the internal anatomical structure remain in registration with the subject's actual internal anatomical structure as the subject moves because of the continual determining, adjusting, and providing.

16. The system of claim 15 , wherein the motion tracking device is configured to capture images of the subject and the computing device is configured to estimate the skeletal pose and position from the captured images.

17. The system of claim 15 , wherein the computing device is further configured to detect gestures made by the subject and to perform in response actions associated with the gestures.

18. The system of claim 15 , wherein the computing device is further configured to quiz the subject as to the anatomical structures.

Assignees

• Univ Of South Florida (A Florida Non-Profit Corporation)
• Univ South Florida

Inventors

• Sun Yu
• Johnson Adrian S

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G09B23/30

  Anatomical models {(G09B23/281 - G09B23/288 take precedence)} · CPC title

• G09B19/003 Primary

  Repetitive work cycles; Sequence of movements · CPC title