Method and system for harvesting lesion annotations

Patent metadata

Field	Value
Publication number	US-11620745-B2
Application number	US-202016984727-A
Country	US
Kind code	B2
Filing date	Aug 4, 2020
Priority date	Jan 17, 2020
Publication date	Apr 4, 2023
Grant date	Apr 4, 2023

Abstract

A method of harvesting lesion annotations includes conditioning a lesion proposal generator (LPG) based on a first two-dimensional (2D) image set to obtain a conditioned LPG, including adding lesion annotations to the first 2D image set to obtain a revised first 2D image set, forming a three-dimensional (3D) composite image according to the revised first 2D image set, reducing false-positive lesion annotations from the revised first 2D image set according to the 3D composite image to obtain a second-revised first 2D image set, and feeding the second-revised first 2D image set to the LPG to obtain the conditioned LPG, and applying the conditioned LPG to a second 2D image set different than the first 2D image set to harvest lesion annotations.

First claim

What is claimed is:

1. A method of harvesting lesion annotations, comprising:
conditioning a lesion proposal generator (LPG) based on a first two-dimensional (2D) image set to obtain a conditioned LPG, including:
adding lesion annotations to the first 2D image set to obtain a revised first 2D image set; forming a three-dimensional (3D) composite image according to the revised first 2D image set; reducing false-positive lesion annotations from the revised first 2D image set according to the 3D composite image to obtain a second-revised first 2D image set; and feeding the second-revised first 2D image set to the LPG to obtain the conditioned LPG;

applying the conditioned LPG to a second 2D image set different than the first 2D image set, to obtain a second set of lesion annotations that includes a second initial set of lesion annotations existing prior to applying the conditioned LPG and a second harvested set of lesion annotations after applying the conditioned LPG,
subjecting the second harvested set of lesion annotations to a lesion proposal classifier (LPC) to obtain a second harvested set of positive lesion annotations and a second harvested set of negative lesion annotations;
obtaining a first set of lesion annotations from the second-revised first 2D image set; and
feeding the first set of lesion annotations, the second initial set of lesion annotations, and the second harvested sets of positive and negative lesion annotations to the conditioned LPG to obtain a re-conditioned LPG for harvesting lesion annotations.

2. The method of claim 1 , wherein reducing false-positive lesion annotations from the revised first 2D image set according to the 3D composite image includes:
adding 2D bounding boxes over annotated lesions in the revised first 2D image set;
stacking the revised first 2D image set;
fusing the 2D bounding boxes after stacking to obtain 3D bounding boxes;
determining if a lesion annotation is a false-positive according to a comparison between the 2D bounding boxes and the 3D bounding boxes; and
marking the lesion annotation as negative to reduce false-positive lesion annotations from the revised first 2D image set.

3. The method of claim 1 , further comprising:
obtaining a first set of lesion annotations from the second-revised first 2D image set and subjecting the first set of lesion annotations to a lesion proposal classifier (LPC) to obtain a first set of positive lesion annotations and a first set of negative lesion annotations; and
feeding the first sets of positive and negative lesion annotations to the LPC to obtain a conditioned LPC.

4. The method of claim 1 , further comprising:
feeding a second set of positive lesion annotations and a second set of negative lesion annotations to a lesion proposal classifier (LPC) to obtain a conditioned LPC.

5. The method of claim 1 , wherein the second 2D image set is smaller in image count than the first 2D image set.

6. A lesion imaging system, comprising a lesion proposal generator (LPG) for harvesting lesion annotations, the LPG including a memory and a processor coupled to the memory, the processor being configured to perform:
conditioning the LPG based on a first two-dimensional (2D) image set to obtain a conditioned LPG, including: adding lesion annotations to the first 2D image set to obtain a revised first 2D image set; forming a three-dimensional (3D) composite image according to the revised first 2D image set; reducing false-positive lesion annotations from the revised first 2D image set according to the 3D composite image to obtain a second-revised first 2D image set; and feeding the second-revised first 2D image set to the LPG to obtain the conditioned LPG;
applying the conditioned LPG to a second 2D image set different than the first 2D image set, to obtain a second set of lesion annotations that includes a second initial set of lesion annotations existing prior to applying the conditioned LPG and a second harvested set of lesion annotations after applying the conditioned LPG,
subjecting the second harvested set of lesion annotations to a lesion proposal classifier (LPC) to obtain a second harvested set of positive lesion annotations and a second harvested set of negative lesion annotations;
obtaining a first set of lesion annotations from the second-revised first 2D image set; and
feeding the first set of lesion annotations, the second initial set of lesion annotations, and the second harvested sets of positive and negative lesion annotations to the conditioned LPG to obtain a re-conditioned LPG for harvesting lesion annotations.

7. The lesion imaging system of claim 6 , wherein reducing false-positive lesion annotations from the revised first 2D image set according to the 3D composite image includes:
adding 2D bounding boxes over annotated lesions in the revised first 2D image set;
stacking the revised first 2D image set;
fusing the 2D bounding boxes after stacking to obtain 3D bounding boxes;
determining if a lesion annotation is a false-positive according to a comparison between the 2D bounding boxes and the 3D bounding boxes; and
marking the lesion annotation as negative to reduce false-positive lesion annotations from the revised first 2D image set.

8. The lesion imaging system of claim 6 , further comprising: a lesion proposal classifier (LPC), wherein the processor is further configured to perform:
obtaining a first set of lesion annotations from the second-revised first 2D image set and subjecting the first set of lesion annotations to the LPC to obtain a first set of positive lesion annotations and a first set of negative lesion annotations; and
feeding the first sets of positive and negative lesion annotations to the LPC to obtain a conditioned LPC.

9. The lesion imaging system of claim 6 , further comprising a lesion proposal classifier (LPC), wherein the processor is further configured to perform:
feeding a second set of positive lesion annotations and a second set of negative lesion annotations to the LPC to obtain a conditioned LPC.

10. A non-transitory computer-readable storage medium storing a plurality of instructions, and when being executed, the plurality of instructions cause a processor to perform:
conditioning a lesion proposal generator (LPG) based on a first two-dimensional (2D) image set to obtain a conditioned LPG, including: adding lesion annotations to the first 2D image set to obtain a revised first 2D image set; forming a three-dimensional (3D) composite image according to the revised first 2D image set; reducing false-positive lesion annotations from the revised first 2D image set according to the 3D composite image to obtain a second-revised first 2D image set; and feeding the second-revised first 2D image set to the LPG to obtain the conditioned LPG;
applying the conditioned LPG to a second 2D image set different than the first 2D image set, to obtain a second set of lesion annotations that includes a second initial set of lesion annotations existing prior to applying the conditioned LPG and a second harvested set of lesion annotations after applying the conditioned LPG,
subjecting the second harvested set of lesion annotations to a lesion proposal classifier (LPC) to obtain a second harvested set of positive lesion annotations and a second harvested set of negative lesion annotations;
obtaining a first set of lesion annotations from the second-revised first 2D image set; and
feeding the first set of lesion annotations, the second initial set of lesion annotations, and the second harvested sets of positive and negative lesion annotations to the conditioned LPG to obtain a re-conditioned LPG for harvesting lesion annotations.

11. The non-transitory comput

Assignees

• Ping An Tech Shenzhen Co Ltd

Inventors

• Cai Jinzheng
• Harrison Adam P
• Yan Ke
• Huo Yuankai
• Lu Le

Classifications

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/30096

  Tumor; Lesion · CPC title