Method and apparatus for predicting an image analysis tool based on user behavior

Patent metadata

Field	Value
Publication number	US-11944481-B1
Application number	US-202117520647-A
Country	US
Kind code	B1
Filing date	Nov 6, 2021
Priority date	Feb 23, 2018
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

This patent provides a method and apparatus to improve image analysis for a user viewing an image comprising multiple structures on a display. An eye tracking system is set up for a user viewing the image on a display, such as using an eye facing camera. An analysis of the eye tracking data and a structure that is being viewed is performed. An image analysis tool, such as an image manipulation tool, an image measurement tool or an image annotation tool, is predicted from a group of image analysis tools and is presented to the user. The user can then use the predicted tool as the user sees fit, such as executing the capabilities of the tool via a mouse.

First claim

The invention claimed is:

1. A method comprising:
performing eye tracking of a user
wherein said eye tracking is performed with an eye facing camera,
wherein eye tracking data is analyzed in near real time,
wherein said user is viewing a display,
wherein said display is displaying an image, and
wherein said image comprises multiple structures;

performing an analysis of eye tracking data
wherein said analysis determines that said user is viewing a first structure of said image;

predicting a first image analysis tool from a group of image analysis tools
wherein said first image analysis tool optimizes image analysis of said first structure of said image;

presenting said first image analysis tool to said user;
performing an analysis of eye tracking data at a subsequent time epoch
wherein said analysis of eye tracking data at said subsequent time epoch determines that said user is viewing a second structure, and
wherein said second structure is different from said first structure;

predicting a second image analysis tool from said group of image analysis tools
wherein said second image analysis tool optimizes image analysis of said second structure, and
wherein said second image analysis tool is different than said first image analysis tool; and

presenting said second image analysis tool to said user.

2. The method of claim 1 further comprising performing an artificial intelligence (AI) classification of said first structure.

3. The method of claim 2 further comprising wherein said first image analysis tool is predicted based on said AI classification of said first structure.

4. The method of claim 1 further comprising wherein said display comprises at least one of the group of: a monitor; a phone; a tablet; and, a head display unit (HDU).

5. The method of claim 1 further comprising wherein said analysis of said eye tracking data comprises an artificial intelligence (AI) algorithm.

6. The method of claim 1 further comprising wherein said predicting said first image analysis tool and said second image analysis tool is based on a personalized AI algorithm.

7. The method of claim 6 further comprising wherein said personalized AI algorithm is trained based on data exclusively from said user.

8. The method of claim 7 further comprising:
wherein user data to train said personalized AI algorithm comprises a user-selected tool for a segmented structure.

9. The method of claim 1 further comprising wherein said first image analysis tool comprises an image manipulation tool and wherein said image manipulation tool alters an appearance of said image on said display.

10. The method of claim 9 further comprising wherein said image manipulation tool comprises at least one of the group of:
a grayscale adjustment tool; and
a color adjustment tool.

11. The method of claim 9 further comprising wherein said image manipulation tool comprises at least one of the group of:
a zoom tool; and
a rotation tool.

12. The method of claim 1 further comprising wherein said image analysis tool comprises an image measurement tool and wherein said image measurement tool performs a measurement on said image.

13. The method of claim 12 further comprising wherein said image measurement tool comprises at least one of the group of:
a ruler;
an area measurement tool; and
volume measurement tool.

14. The method of claim 12 further comprising wherein said image measurement tool comprises at least one of the group of:
a pixel value measurement tool; and
a voxel value measurement tool.

15. The method of claim 1 further comprising wherein said first image analysis tool comprises an image mark up tool and wherein said image mark up tool performs an annotation on said image.

16. The method of claim 15 further comprising wherein said image mark up tool comprises at least one of the group of:
an arrow; and
a circle.

17. The method of claim 1 further comprising wherein said first image analysis tool is controlled by a computer mouse.

18. The method of claim 1 further comprising wherein said first image analysis tool comprises an image diagnostic tool comprising at least one of the group of:
a texture analysis tool;
a sub-volume analysis tool;
a feature classification tool; and
an artificial intelligence (AI) analysis tool.

19. A non-transitory computer readable medium having computer readable code thereon for image processing, the computer readable code including instructions comprising:
performing eye tracking of a user
wherein said eye tracking is performed with an eye facing camera,
wherein eye tracking data is performed in near real time,
wherein said user is viewing a display,
wherein said display is displaying an image, and
wherein said image comprises multiple structures;

performing an analysis of eye tracking data at a first time epoch
wherein said analysis determines that said user is viewing a first structure of said image;

predicting a first image analysis tool from a group of image analysis tools based on said first structure
wherein said first image analysis tool optimizes image analysis of said first structure of said image;

presenting said first image analysis tool to said user;
performing an analysis of eye tracking data at a subsequent time epoch
wherein said analysis of eye tracking data at said subsequent time epoch determines that said user is viewing a second structure, and
wherein said second structure is different from said first structure;

predicting a second image analysis tool from said group of image analysis tools based on said second structure
wherein said second image analysis tool optimizes image analysis of said second structure, and
wherein said second image analysis tool is different than said first image analysis tool; and

presenting said second image analysis tool to said user.

20. An apparatus comprising:
an IO device; and
an image processor in communication with the IO device, the image processor comprising a program stored on a computer-readable non-transitory media, the program comprising instructions that:
perform eye tracking of a user
wherein said eye tracking is performed with an eye facing camera,
wherein eye tracking data is performed in near real time,
wherein said user is viewing a display,
wherein said display is displaying an image, and
wherein said image comprises multiple structures;

perform an analysis of eye tracking data at a first time epoch
wherein said analysis determines that said user is viewing a first structure of said image;

predict a first image analysis tool from a group of image analysis tools based on said first structure and said analysis of eye tracking data at said first time epoch
wherein said first image analysis tool optimizes image analysis of said first structure of said image;

present said first image analysis tool to said user;
perform an analysis of eye tracking data at a subsequent time epoch
wherein said analysis of eye tracking data at said subsequent time epoch determines that said user is viewing a second structure, and
wherein said second structure is different from said first structure;

predict a second image analysis tool from said group of image analysis tools based on said second structure and said analysis of eye tracking data at said subsequent time epoch
wherein said second image analysis tool optimizes image analysis of said second structure, and
wherein said second image analysis tool is different than said first image analysis tool;

Assignees

• Douglas Robert Edwin
• Douglas David Byron
• Douglas Kathleen Mary

Inventors

• Douglas Robert Edwin
• Douglas David Byron
• Douglas Kathleen Mary

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• A61B6/5223 Primary

  generating planar views from image data, e.g. extracting a coronal view from a 3D image · CPC title

• A61B6/032

  Transmission computed tomography [CT] · CPC title

• A61B6/466

  adapted to display 3D data · CPC title