Automated selection of an optimal image from a series of images

Patent metadata

Field	Value
Publication number	US-10909681-B2
Application number	US-201916239303-A
Country	US
Kind code	B2
Filing date	Jan 3, 2019
Priority date	Jan 3, 2019
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

A method for identification of an optimal image within a sequence of image frames includes inputting the sequence of images into a computer processor configured for executing a plurality of neural networks and applying a sliding window to the image sequence to identify a plurality of image frame windows. The image frame windows are processed using a first neural network trained to classify the image frames according to identified spatial features. The image frame windows are also processed using a second neural network trained to classify the image frames according to identified serial features. The results of each classification are concatenated to separate each of the image frame windows into one of two classes, one class containing the optimal image. An output is generated to display image frame windows classification as including the optimal image.

First claim

The invention claimed is:

1. A method for identification of an optimal image within a sequence of image frames, comprising:
inputting the sequence of image frames into a computer processor configured for executing a plurality of neural networks;
applying a sliding window to the sequence of image frames to identify a plurality of image frame windows within the sequence;
processing the plurality of image frame windows using a first neural network of the plurality, the first neural network trained for identifying spatial features within the image frames for first classifying the image frame window into spatial classes according to the identified spatial features;
processing the plurality of image frame windows using a second neural network of the plurality, the second neural network trained for identifying serial features among the image frames for second classifying the image frame windows into series classes according to the identified serial features;
concatenating the results of the first classifying and second classifying to separate each of the plurality of image frame windows into one of two classes, wherein image frame windows that include the optimal image are classified into one of the classes; and
generating an output displaying image frame windows that include the optimal image.

2. The method of claim 1 , wherein the sequence of image frames is an MRI time sequence and the serial features comprise time.

3. The method of claim 2 , wherein the MRI time sequence comprises a T1 mapping sequence.

4. The method of claim 1 , wherein the sequence of image frames is a stack of MRI slices and the serial features comprise location within the stack.

5. The method of claim 1 , wherein the first neural network is a convolutional neural network (CNN).

6. The method of claim 5 , wherein the CNN is VGG19.

7. The method of claim 1 , wherein the second neural network is a recurrent neural network (RNN).

8. The method of claim 7 , wherein the RNN is formed from layers comprising long short-term memory (LSTM) models.

9. The method of claim 8 , wherein the LSTM models comprise a three serial bidirectional LSTM.

10. A method for identification of an optimal image within a sequence of image frames, comprising:
inputting the sequence of image frames into a computer processor configured for executing a plurality of neural networks;
applying a sliding window to the sequence of image frames to identify a plurality of image frame windows within the sequence;
processing the plurality of image frame windows using a convolutional neural network (CNN), the CNN trained for identifying spatial features within the image frames for first classifying the image frame window into spatial classes according to the identified spatial features;
processing the plurality of image frame windows using a recurrent neural network (RNN), the RNN trained for identifying serial features among the image frames for second classifying the image frame windows into series classes according to the identified serial features;
concatenating the outputs of the CNN and the RNN to separate each of the plurality of image frame windows into one of two classes, wherein image frame windows that include the optimal image are classified into one of the classes; and
generating an output displaying image frame windows that include the optimal image.

11. The method of claim 10 , wherein the sequence of image frames is an MRI time sequence and the serial features comprise time.

12. The method of claim 11 , wherein the MRI time sequence comprises a T1 mapping sequence.

13. The method of claim 10 , wherein the sequence of image frames is a stack of MRI slices and the serial features comprise location within the stack.

14. The method of claim 10 , wherein the CNN is VGG19.

15. The method of claim 10 , wherein the RNN is formed from layers comprising long short-term memory (LSTM) models.

16. The method of claim 15 , wherein the LSTM models comprise a three serial bidirectional LSTM.

17. A system for identifying an optimal image within a sequence of image frames, the system comprising:
at least one computer processor configured to acquire the sequence of image frames and apply a sliding window to identify a plurality of image frame windows within the sequence;
a spatial classification module configured for identifying spatial features within the image frames for first classifying the image frame windows into spatial classes according to the identified spatial features:
a temporal/sequential classification module configured for identifying serial features among the image frames for second classifying the image frame windows into series classes according to the identified serial features;
a concatenator module configured for combining the results of the first classifying and second classifying to separate each of the plurality of image frame windows into one of two classes, wherein image frame windows that include the optimal image are classified into one of the classes; and
a display module configured for generating an output display of the image frame windows that include the optimal image.

18. The system of claim 17 , wherein the sequence of image frames is an MRI time sequence and the serial features comprise time.

19. The system of claim 18 , wherein the MRI time sequence comprises a T1 mapping sequence.

20. The system of claim 17 , wherein the sequence of image frames is a stack of MRI slices and the serial features comprise location within the stack.

21. The system of claim 17 , wherein the spatial classification module and the temporal/sequential classification module are neural networks.

22. The system of claim 21 , wherein the spatial classification module is a convolutional neural network (CNN).

23. The system of claim 21 , wherein temporal/sequential classification module is a recurrent neural network (RNN).

24. The system of claim 23 , wherein the RNN is formed from layers comprising long short-term memory (LSTM) models.

25. The system of claim 24 , wherein the LSTM models comprise a three serial bidirectional LSTM.

Assignees

• Univ California

Inventors

• Hsiao Albert
• Bahrami Naeim
• Retson Tara

Classifications

• G06N20/20 Primary

  Ensemble learning · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T7/0016 Primary

  involving temporal comparison · CPC title

• G06F18/241

  relating to the classification model, e.g. parametric or non-parametric approaches · CPC title