Image registration device, image registration method, and ultrasonic diagnosis apparatus having image registration device

Patent metadata

Field	Value
Publication number	US-9710910-B2
Application number	US-201514753394-A
Country	US
Kind code	B2
Filing date	Jun 29, 2015
Priority date	Sep 30, 2014
Publication date	Jul 18, 2017
Grant date	Jul 18, 2017

Abstract

There is provided an image registration device and an image registration method. The device includes: a feature extractor configured to extract, from a first image, a first feature group and to extract, from a second image, a second feature group; a feature converter configured to convert, using a converted neural network in which a correlation between features is learned, the extracted second feature group to correspond to the extracted first feature group, to obtain a converted group; and a register configured to register the first image and the second image based on the converted group and the extracted first feature group.

First claim

What is claimed is:

1. An image registration device, comprising:
a feature extractor configured to extract, from a first medical image, a first feature group and to extract, from a second medical image, a second feature group;
a feature converter configured to convert, using a converted neural network in which a correlation between features is learned, the extracted second feature group to correspond to the extracted first feature group, to obtain a converted group; and
a register configured to register the first medical image and the second medical image based on the converted group and the extracted first feature group.

2. The device according to claim 1 , wherein the register is configured to determine a registration function based on an error between the converted group and the extracted second feature group, and to register the first medical image and the second medical image by applying the determined registration function.

3. The device according to claim 1 , wherein the converted neural network is trained without supervision based on the registered first medical image and the registered second medical image.

4. The device according to claim 3 , wherein the converted neural network is trained without supervision according to a relation between features extracted from a same part of the registered image.

5. The device according to claim 4 , wherein the converted neural network has a multilayer perceptron structure.

6. The device according to claim 1 , further comprising:
a first feature neural network in which extraction of the first feature group is learned without supervision using a plurality of first medical images corresponding to a first modality of a medical device; and
a second feature neural network in which extraction of the second feature group is learned without supervision using a plurality of second medical images corresponding to a second modality of another medical device.

7. The device according to claim 6 , wherein the feature extractor is configured to extract features of the first medical image and the second medical image based on the first feature neural network and the second feature neural network, respectively.

8. The device according to claim 1 , wherein the first medical image is a medical image obtained in real time, and the second medical image is a pre-obtained medical image.

9. The device according to claim 7 , wherein the first medical image is an ultrasound image, and the second medical image is an image of a computed tomography image, a magnetic resonance image, a positron emission tomography image, or a single photon emission computed tomography image.

10. The device according to claim 1 , wherein the first medical image is a pre-obtained medical image, and the second medical image is obtained in real time.

11. The device according to claim 1 , wherein the first medical image and the second medical image are obtained by medical devices having different modalities.

12. An image registration method, comprising:
extracting, from a first medical image, a first feature group and extracting, from a second medical image, a second feature group;
converting, using a converted neural network in which a correlation between features is learned, the extracted first feature group to correspond to the extracted second feature group, to obtain a converted group; and
registering the first medical image and the second medical image based on the converted group and the extracted second feature group.

13. The method according to claim 12 , wherein the registering comprises:
determining a registration function based on an error between the converted group and the extracted second feature group; and
applying the determined registration function to one of the first medical image and the second medical image.

14. The method according to claim 12 , wherein the extracting comprises:
extracting the first feature group using a first feature neural network in which feature extraction is learned without supervision using a plurality of first medical images corresponding to a first modality of a medical device; and
extracting the second feature group using a second feature neural network in which feature extraction is learned without supervision using a plurality of second medical images corresponding to a second modality of another medical device.

15. The method according to claim 14 , wherein the first feature neural network, the second feature neural network, and the converted neural network each have a multilayer perceptron structure.

16. An ultrasonic diagnosis apparatus, comprising:
an image obtainer configured to obtain an ultrasound image of a subject;
a feature extractor configured to extract, from a medical image obtained in advance by imaging the subject, a first feature group and to extract, from the obtained ultrasound image, a second feature group;
a feature converter configured to convert, using a converted neural network in which a correlation between features is learned, the extracted first feature group and the extracted second feature group into a same feature space; and
a register configured to register the obtained ultrasound image and the obtained medical image based on the first feature group and the second feature group converted into the same feature space.

17. The apparatus according to claim 16 , wherein the first feature group is generated by a first feature neural network that is trained without supervision in advance based on a plurality of medical images, and the second feature group is generated by a second feature neural network that is trained without supervision in advance based on a plurality of medical images.

18. The apparatus according to claim 16 , further comprising a communicator configured to receive the medical image from an external device.

19. The apparatus according to claim 16 , wherein the register is configured to determine a registration function based on an error between the first feature group and the second feature group converted into the same feature space, to apply the determined registration function to the medical image, and to register the medical image and the ultrasound image.

20. The apparatus according to claim 16 , wherein the converted neural network is trained with supervision based on the registered ultrasound image and the registered medical image.

21. An image registration device, comprising:
a feature converter configured to obtain a converted group by converting, using a converted neural network in which a correlation between features is learned, a second feature group to correspond to a first feature group, the second feature group obtained from a second medical image corresponding to a second modality of a medical device and the first feature group obtained from a first medical image corresponding to a first modality of another medical device; and
a register configured to register the first medical image and the second medical image based on the converted group and the first feature group.

22. The device according to claim 21 , wherein the register is configured to determine a registration function based on an error between the converted group and the second feature group, and to register the first medical image and the second medical image by applying the determined registration function.

23. The device according to claim 21 , wherein the converted neural network is trained without supervision based on the registered first medical image and the registered sec

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Sunkwon
• Chang Jungwoo
• Bang Won-Chul

Classifications

• G06F18/217

  Validation; Performance evaluation; Active pattern learning techniques · CPC title

• G06V10/20 Primary

  Image preprocessing · CPC title

• G06F18/2178

  based on feedback of a supervisor · CPC title

• G06V10/7515

  Shifting the patterns to accommodate for positional errors · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title