Interactive iterative image annotation

Patent metadata

Field	Value
Publication number	US-2022019860-A1
Application number	US-201917294731-A
Country	US
Kind code	A1
Filing date	Nov 15, 2019
Priority date	Nov 22, 2018
Publication date	Jan 20, 2022
Grant date	—

Abstract

A system and computer-implemented method are provided for annotation of image data. A user is enabled to iteratively annotate the image data. An iteration of said iterative annotation comprises generating labels for a current image data part based on user-verified labels of a previous image data part, and enabling the user to verify and correct said generated labels to obtain user-verified labels for the current image data part. The labels for the current image data part are generated by combining respective outputs of a label propagation algorithm and a machine-learned classifier trained on user-verified labels and image data and applied to image data of the current image data part. The machine-learned classifier is retrained using the user-verified labels and the image data of the current image data part to obtain a retrained machine-learned classifier.

First claim

1 . A system for annotation of image data, the system comprising:
an input interface configured to access the image data to be annotated; a user interface subsystem comprising:
a user input interface configured to receive user input data from a user input device operable by a user;
a display output interface configured to provide display data to a display to visualize output of the system;
a processor configured to, using the user interface subsystem, establish a user interface which enables the user to iteratively annotate the image data, wherein an iteration of said iterative annotation comprises:
the processor generating labels for a current image data part based on user-verified labels of a previous image data part;
via the user interface, enabling the user to verify and correct said generated labels to obtain user-verified labels for the current image data part;
wherein the processor is further configured to:
generate the labels for the current image data part by combining, by weighting, respective outputs of:
a label propagation algorithm which propagates the user-verified labels of the previous image data part to the current image data part, and
a machine-learned classifier for labeling of image data, wherein the machine-learned classifier is trained on user-verified labels and image data and applied to image data of the current image data part; and

retrain the machine-learned classifier using the user-verified labels and the image data of the current image data part to obtain a retrained machine-learned classifier.

2 . The system according to claim 1 , wherein the processor is configured to retrain the machine-learned classifier between iterations of the iterative annotation, or between the iterative annotation of different image data.

3 . The system according to claim 1 , wherein the processor is configured to adjust the weighting during the iterative annotation of the image data, or between the iterative annotation of different image data.

4 . The system according to claim 1 , wherein the processor is configured to determine the weighting based on a metric quantifying an annotation accuracy of the label propagation algorithm and/or the machine-learned classifier.

5 . The system according to claim 4 , wherein the metric quantifies the annotation accuracy based on a difference between i) the output of the label propagation algorithm and/or the output of the machine-learned classifier and ii) the user-corrected labels.

6 . The system ( 100 ) according to claim 3 , wherein the processor is configured to adjust the weighting by increasing a weighting of the output of the machine-learned classifier relative to the output of the label propagation algorithm.

7 . The system according to claim 6 , wherein the processor is configured to start the weighting of the output of the machine-learned classifier at or substantially at zero at a start of the iterative annotation.

8 . The system according to claim 1 , wherein the weighting comprises a global weighting per image data part and/or a local weighting per pixel, voxel or other image sub-region.

9 . The system according to claim 1 , wherein the output of the label propagation algorithm and/or the output of the machine-learned classifier is a probability map, or one or more control points defining a contour.

10 . The system according to claim 1 , wherein the label propagation algorithm is configured to propagate the user-verified labels of the previous image data part to the current image data part based on a similarity in image data between the previous image data part and the current image data part.

11 . The system according to claim 10 , wherein the label propagation algorithm is a patch-based label propagation algorithm.

12 . A workstation or imaging apparatus comprising the system according to claim 1 .

13 . A computer-implemented method for annotation of image data, the method comprising:
accessing the image data to be annotated; using a user interface, enabling a user to iteratively annotate the image data, wherein an iteration of said iterative annotation comprises:
generating labels for a current image data part based on user-verified labels of a previous image data part;
via the user interface, enabling the user to verify and correct said generated labels to obtain user-verified labels for the current image data part;
wherein the labels for the current image data part are generated by combining, by weighting, respective outputs of:
a label propagation algorithm which propagates the user-verified labels of the previous image data part to the current image data part, and
a machine-learned classifier for labeling of image data, wherein the machine-learned classifier is trained on user-verified labels and image data and applied to image data of the current image data part; and
retraining the machine-learned classifier using the user-verified labels and the image data of the current image data part to obtain a retrained machine-learned classifier.

14 . (canceled)

15 . A non-transitory computer readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:
access image data to be annotated; using a user interface, enable a user to iteratively annotate the image data, wherein an iteration of said iterative annotation comprises:
generating labels for a current image data part based on user-verified labels of a previous image data part;
via the user interface, enabling the user to verify and correct said generated labels to obtain user-verified labels for the current image data part;
wherein the labels for the current image data part are generated by combining, by weighting, respective outputs of:
a label propagation algorithm which propagates the user-verified labels of the previous image data part to the current image data part, and
a machine-learned classifier for labeling of image data, wherein the machine-learned classifier is trained on user-verified labels and image data and applied to image data of the current image data part; and

retraining the machine-learned classifier using the user-verified labels and the image data of the current image data part to obtain a retrained machine-learned classifier.

16 . The non-transitory computer readable medium of claim 15 , wherein the machine-learned classifier is retrained between iterations of the iterative annotation, or between the iterative annotation of different image data.

17 . The non-transitory computer readable medium of claim 15 , wherein weighting is adjusted during the iterative annotation of the image data or between the iterative annotation of different image data.

18 . The non-transitory computer readable medium of claim 15 , wherein the weighting is determined based on a metric quantifying an annotation accuracy of the label propagation algorithm and/or the machine-learned classifier.

19 . The non-transitory computer readable medium of claim 18 , wherein the metric quantifies the annotation accuracy based on a difference between i) the output of the label propagation algorithm and/or the output of the machine-learned classifier and ii) the user-corrected labels.

20 . The non-transitory computer readable medium of claim 17 , wherein the weighting is adjusted by increasing a weighting of the output of the machine-learned classifier relative to the output of the label propagation algorithm.

21 . The non-transitory computer readable medium of claim 20 , wherein the we

Assignees

• Koninklijke Philips Nv

Inventors

• Weese Juergen
• Blaffert Thomas
• Brosch Tom
• Barschdorf Hans

Classifications

• G06F18/40

  Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor · CPC title

• G06F18/2178 Primary

  based on feedback of a supervisor · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06V2201/03

  Recognition of patterns in medical or anatomical images · CPC title

• G06N20/00

  Machine learning · CPC title