Method for positioning key features of a lens based on ocular B-mode ultrasound images

Patent metadata

Field	Value
Publication number	US-11723630-B2
Application number	US-202117301615-A
Country	US
Kind code	B2
Filing date	Apr 9, 2021
Priority date	May 14, 2020
Publication date	Aug 15, 2023
Grant date	Aug 15, 2023

Abstract

A method for positioning key features of a lens based on ocular B-mode ultrasound images includes: acquiring and preprocessing the ocular B-mode ultrasound images to obtain a preprocessed B-mode ultrasound image, eyeball coordinates and lens coordinates; sending the preprocessed B-mode ultrasound image, the eyeball coordinates and the lens coordinates into a trained target detection network YOLOv3 to obtain eyeball position images and lens position images; substituting the eyeball position images and the lens position images into a trained feature extraction network group to obtain image features and feature coordinates corresponding to the eyeball position images and the lens position images, respectively; substituting the image features into a trained collaborative learning network to screen key image features; and marking a feature coordinate corresponding to the key image features on the ocular B-mode ultrasound images to complete positioning the key features of the lens.

First claim

What is claimed is:

1. A method for positioning key features of a lens based on ocular B-mode ultrasound images, comprising:
S 1 : acquiring and preprocessing the ocular B-mode ultrasound images to obtain a preprocessed B-mode ultrasound image, eyeball coordinates and lens coordinates;
S 2 : fusing the preprocessed B-mode ultrasound image and the eyeball coordinates and then sending the preprocessed B-mode ultrasound image and the eyeball coordinates into a trained target detection network to obtain eyeball position images; sending the preprocessed B-mode ultrasound image and the lens coordinates into the trained target detection network to obtain lens position images;
S 3 : substituting the eyeball position images and the lens position images into a trained feature extraction network group to obtain image features and feature coordinates corresponding to the eyeball position images and the lens position images, respectively;
S 4 : substituting the image features into a trained collaborative learning network to screen key image features; and
S 5 : marking a feature coordinate corresponding to the key image feature on the ocular B-mode ultrasound images to complete positioning the key features of the lens.

2. The method according to claim 1 , wherein a method of preprocessing the ocular B-mode ultrasound images comprises:
S 1 - 1 : converting the ocular B-mode ultrasound images into image files of a set size and a set format to obtain an image file set;
S 1 - 2 : dividing the image file set into an image file subset of a target detection group for training a target detection network and an image file subset of a feature extraction group for training a feature extraction network; and
S 1 - 3 : marking the eyeball coordinates and the lens coordinates in the image file subset of the target detection group.

3. The method according to claim 2 , wherein the eyeball coordinates comprise coordinates of an upper left corner and a lower right corner of an eyeball; the lens coordinates comprise coordinates of an upper left corner and a lower right corner of the lens.

4. The method according to claim 3 , wherein a specific step of training the target detection network comprises:
S 2 - 1 : normalizing the eyeball coordinates, and adjusting the preprocessed B-mode ultrasound image to a target size to obtain an adjusted B-mode ultrasound image;
S 2 - 2 : sending the adjusted B-mode ultrasound image into a backbone network to obtain a feature dataset comprising a plurality of modules;
S 2 - 3 : splicing the plurality of modules in the feature dataset with each other to obtain splicing results corresponding to the plurality of modules;
S 2 - 4 : performing a convolution processing on the splicing results to obtain possible eyeball positions;
S 2 - 5 : substituting the possible eyeball positions into the target detection network to enable the target detection network to use feature maps of three eyeball position predictions to obtain the eyeball position images; and
S 2 - 6 : repeatedly performing steps S 2 - 1 to S 2 - 5 on the lens coordinates to obtain the lens position images.

5. The method according to claim 1 , wherein a specific step of obtaining the trained feature extraction network group comprises:
S 3 - 1 : dividing the eyeball position images and the lens position images into training sets, validation sets and test sets, respectively;
S 3 - 2 : separately sending the training sets corresponding to the eyeball position images and the lens position images into the feature extraction network separately corresponding to the eyeball position images and the lens position images for training to separately obtain a trained feature extraction network;
S 3 - 3 : separately sending the validation sets corresponding to the eyeball position images and the lens position images into the trained feature extraction network separately corresponding the eyeball position images and the lens position images to obtain lens features corresponding to the eyeball position images and the lens position images;
S 3 - 4 : evaluating states of the lens according to the lens features, comparing evaluation results with real labels in the validation sets, and recording a validation accuracy rate;
S 3 - 5 : repeatedly performing steps S 3 - 2 to step S 3 - 4 according to a set number of times, saving weights of two feature extraction networks with a highest validation accuracy rate during a repeated performing process, taking the two feature extraction networks as a target feature extraction network group, and substituting the weights into the test sets to obtain a test accuracy rate of the target feature extraction network group; and
S 3 - 6 : comparing the test accuracy rate with a target value, when the test accuracy rate is less than the target value, repeatedly performing step S 3 - 5 until the test accuracy rate of the target feature extraction network group is greater than or equal to the target value, and using the target feature extraction network group corresponding to the test accuracy rate as the trained feature extraction network group.

6. The method according to claim 5 , wherein the feature extraction network comprises an eyeball module corresponding to the eyeball position images and a lens module corresponding to the lens position images.

7. The method according to claim 6 , wherein each of the eyeball module and the lens module comprises a back propagation neural network BPNN, wherein the back propagation neural network is trained by a convolutional neural network, a Fourier descriptor, and a gray-level co-occurrence matrix.

8. The method according to claim 7 , wherein a specific step of training the feature extraction network comprises:
S 3 - 2 - 1 : adjusting the eyeball position images and the lens position images to a required size, and performing a deformation processing to obtain deformed position images;
S 3 - 2 - 2 : substituting the deformed position images into the Fourier descriptor to obtain 36-dimensional shape features corresponding to the deformed position images;
S 3 - 2 - 3 : substituting the deformed position images into the gray-level co-occurrence matrix, and calculating energies, contrasts, entropies, and inverse differences of the gray-level co-occurrence matrix in four directions to obtain 16-dimensional texture features corresponding to the position images;
S 3 - 2 - 4 : adopting an image dataset to pre-train the convolutional neural network to obtain a trained convolutional neural network, substituting the deformed position images into the trained convolutional neural network, and obtaining and using 2208-dimensional features of a penultimate layer of the trained convolutional neural network corresponding to the position images as depth features; and
S 3 - 2 - 5 : fusing and then substituting the 36-dimensional shape features, the 16-dimensional texture features and the depth features into the back propagation neural network BPNN for training to obtain a trained back propagation neural network BPNN as the trained feature extraction network.

9. The method according to claim 8 , wherein the deformation processing comprises random horizontal flip, random rotation of 0-10 degrees, brightness adjustment, color jitter, and normalization.

10. The method according to claim 7 , wherein a step of obtaining the trained collaborative learning network comprises:
S 5 - 1 : using a learning layer of the back propagation neural network BPNN in each of the eyeball module and the lens module as an input layer of a collaborative learning network;
S 5 - 2 : adding two fully connected layers behind the input layer of the collaborative learning network, wherein, the two fully connected layers are used as a

Assignees

• Univ Sichuan

Inventors

• Lv Jiancheng
• Wang Yong
• Sang Yongsheng
• Peng Dezhong
• Peng Xi
• Sun Yanan
• He Zhenan
• Ye Qing
• Li Mao
• Liu Quanhui

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• A61B8/5246 Primary

  combining images from the same or different imaging techniques, e.g. color Doppler and B-mode · CPC title

• A61B8/10

  Eye inspection · CPC title

• A61B8/5223

  for extracting a diagnostic or physiological parameter from medical diagnostic data (for algorithms to analyse biomedical images G06T7/0012) · CPC title