Eye-tracking image viewer for digital pathology

Patent metadata

Field	Value
Publication number	US-11747900-B2
Application number	US-202217859765-A
Country	US
Kind code	B2
Filing date	Jul 7, 2022
Priority date	Dec 19, 2018
Publication date	Sep 5, 2023
Grant date	Sep 5, 2023

Abstract

Image viewing in digital pathology using eye-tracking. In an embodiment, a position of a user's gaze on a graphical user interface, comprising at least a portion of a digital slide image within a macro view, is repeatedly detected based on an output from an eye-tracking device. After detecting a change of the user's gaze from a first position to a second position on the graphical user interface, a view of the digital slide image within the macro view is automatically panned based on the second position, so as to move a position on the digital slide image that corresponds to the second position on the graphical user interface toward a center of the macro view.

First claim

What is claimed is:

1. A system comprising:
a computing device configured to sense a position of a user's gaze on a display; and
at least one hardware processor configured to:
detect the position of the user's gaze on the display, based on an output from the computing device, wherein the display displays at least a portion of an image within a macro view and a thumbnail image of the image;
determine that the detected position of the user's gaze is within one of the thumbnail image or the macro view; and
in response to determining that the detected position of the user's gaze is within one of the thumbnail image or the macro view,
determine a position on the image that corresponds to the detected position within one of the thumbnail image or the macro view, and
automatically update a view of the image within the macro view to move the position on the image that corresponds to the detected position within one of the thumbnail image or the macro view toward a center of the macro view.

2. The system of claim 1 , wherein to automatically update the view of the image, the at least one hardware processor is further configured to automatically pan the view of the image.

3. The system of claim 1 , wherein the at least one hardware processor is further configured to determine a modification of the position of the user's gaze from a first position to a second position, wherein the at least one hardware processor is further configured to automatically update the view of the image in response to determining the modification of the position of the user's gaze.

4. The system of claim 1 , wherein the at least one hardware processor is further configured to determine a modification of the position of the user's gaze from a first position to a second position, wherein the at least one hardware processor is further configured to automatically update the view of the image to move a prior position away from the center of the macro view in response to determining the modification of the position of the user's gaze, wherein the prior position corresponds to the first position and the position corresponds to the second position.

5. The system of claim 1 , wherein the at least one hardware processor is further configured to determine a modification of the position of the user's gaze from a first position to a second position, wherein the at least one hardware processor is further configured to automatically update the view of the image in response to determining the modification of the position of the user's gaze exceeds a threshold value.

6. The system of claim 1 , wherein the image is obtained from a digital pathology slide server or scanning apparatus over a wired or wireless network.

7. The system of claim 1 , wherein the thumbnail image is associated with a first image resolution and the macro view is associated with a second image resolution greater than the first image resolution.

8. The system of claim 1 , wherein the thumbnail image is associated with a first magnification and the macro view is associated with a second magnification greater than the first magnification.

9. The system of claim 1 , wherein the thumbnail image is overload on the macro view on the display.

10. The system of claim 1 , wherein the at least one hardware processor is further configured to:
determine a modification of the position of the user's gaze from a first position to a second position; and
determine a speed of automatic updating based on the modification, wherein the at least one hardware processor is further configured to automatically update the view of the image based on the speed of automatic updating.

11. The system of claim 1 , wherein the at least one hardware processor is further configured to:
determine an orientation of a user's head based on the output from the computing device; and
determine a starting gesture based on the orientation of the user's head, wherein the at least one hardware processor is further configured to automatically update the view of the image further in response to determining the starting gesture.

12. The system of claim 11 , wherein the at least one hardware processor is further configured to:
determine a stopping gesture based on the orientation of the user's head, and
in response to determining the stopping gesture, automatically stop updating the view of the image.

13. A non-transitory computer-readable medium storing computer-executable instructions that, when executed by one or more computing devices, cause the one or more computing devices to:
detect a position of a user's gaze on a display, wherein the display displays at least a portion of an image within a macro view and a thumbnail image of the image;
determine that the detected position of the user's gaze is within one of the thumbnail image or the macro view; and
in response to determining that the detected position of the user's gaze is within one of the thumbnail image or the macro view,
determine a position on the image that corresponds to the detected position within one of the thumbnail image or the macro view, and
automatically update a view of the image within the macro view to move the position on the image that corresponds to the detected position within one of the thumbnail image or the macro view toward a center of the macro view.

14. The non-transitory computer-readable medium of claim 13 , the non-transitory computer-readable medium storing further computer-executable instructions that, when executed by the one or more computing devices, cause the one or more computing devices to:
determine a modification of the position of the user's gaze from a first position to a second position; and
automatically update the view of the image in response to determining the modification of the position of the user's gaze.

15. The non-transitory computer-readable medium of claim 13 , the non-transitory computer-readable medium storing further computer-executable instructions that, when executed by the one or more computing devices, cause the one or more computing devices to:
determine a modification of the position of the user's gaze from a first position to a second position; and
automatically update the view of the image to move a prior position away from the center of the macro view in response to determining the modification of the position of the user's gaze, wherein the prior position corresponds to the first position and the position corresponds to the second position.

16. The non-transitory computer-readable medium of claim 13 , the non-transitory computer-readable medium storing further computer-executable instructions that, when executed by the one or more computing devices, cause the one or more computing devices to:
determine a modification of the position of the user's gaze from a first position to a second position; and
automatically update the view of the image in response to determining the modification of the position of the user's gaze exceeds a threshold value.

17. The non-transitory computer-readable medium of claim 13 , wherein the thumbnail image and the macro view are associated with at least one of different image resolutions or different magnifications.

18. The non-transitory computer-readable medium of claim 13 , the non-transitory computer-readable medium storing further computer-executable instructions that, when executed by the one or more computing devices, cause the one or more computing devices to:
determine an orientation of a user's head based on the output from the computing device;
determine a starting gesture based on the orient

Assignees

• Leica Biosystems Imaging Inc

Inventors

• Olson Allen
• Saligrama Kiran

Classifications

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0485

  Scrolling or panning · CPC title

• G06V40/18

  Eye characteristics, e.g. of the iris · CPC title

• G16H30/20

  for handling medical images, e.g. DICOM, HL7 or PACS · CPC title