Disease recognition from images having a large field of view

Patent metadata

Field	Value
Publication number	US-10755129-B2
Application number	US-201916554485-A
Country	US
Kind code	B2
Filing date	Aug 28, 2019
Priority date	Oct 5, 2017
Publication date	Aug 25, 2020
Grant date	Aug 25, 2020

Abstract

In an embodiment, a computer-implemented method of detecting infected objects from large field-of-view images is disclosed. The method comprises receiving, by a processor, a digital image capturing multiple objects; generating, by the processor, a plurality of scaled images from the digital image respectfully corresponding to a plurality of scales; and computing a group of feature matrices for the digital image. The method further comprises, for each of the plurality of scaled images. selecting a list of candidate regions from the scaled image each likely to capture a single object; and for each of the list of candidate regions, performing the following steps: mapping the candidate region back to the digital image to obtain a mapped region; identifying a corresponding portion from each of the group of feature matrices based on the mapping; and determining whether the candidate region is likely to capture the single object infected with a disease based on the group of corresponding portions. In addition, the method comprises choosing a group of final regions from the lists of mapped regions based on the determining; and causing display of information regarding the group of final regions.

First claim

What is claimed is:

1. A computer-implemented method of detecting infected objects from large field-of-view images, comprising:
receiving, by a processor, a first set of training images,
at least a certain image of the first set of training images capturing an object of a certain type in more than half of a total area of the certain image;

building, by the processor, a first digital model based on the first set of training images,
the first digital model configured to receive a list of feature vectors of a first input image and produce a determination of whether the first input image captures an object of the certain type;

receiving a second set of training images,
for each of a plurality of classes, at least a specific image of the second set of training images capturing an object of the certain type that is associated with the class in more than half of a total area of the specific image;

augmenting, for a plurality of scales, the second set of training images with copies of a group of the second set of training images resized at the plurality of scales to generate an augmented set of training images including the second set of training images;
creating a second digital model based on the augmented set of training images,
the second digital model configured to receive a second input image capturing a plurality of objects of the certain type and produce a determination of whether a region of the second input image captures an object of the certain type associated with any of the plurality of classes;

receiving a third input image capturing multiple objects of the certain type;
identifying a plurality of regions from the third input image using the first digital model;
computing a group of feature matrices for the third input image using the second digital model;
mapping a candidate region of the plurality of regions back to the third input image to obtain a mapped region;
identifying a corresponding portion from each of the group of feature matrices based on the mapping; and
determining whether the candidate region is likely to capture an object of the certain type associated with one of the plurality of classes using the second digital model;
causing display of information related to the determining.

2. The computer-implemented method of claim 1 , wherein the first set of training images comprises:
a first subset of images labeled as positive samples, where, for each specific positive image of the first subset of images, an object of the certain type is captured in more than half of a total area of the specific positive image; and
a second subset of images labeled as negative samples, where, for each specific negative image of the second subset of images, two or more objects are captured in the specific negative image, one object is captured in less than half of the total area of the specific negative image, or no objects are captured in the specific negative image.

3. The computer-implemented method of claim 1 , wherein the first digital model implements a support vector machine (SVM) model configured to determine whether the first input image captures the object of the certain type.

4. The computer-implemented method of claim 1 , wherein the list of feature vectors of the first input image comprise a histogram of oriented gradients (HOG) value for the first input image.

5. The computer-implemented method of claim 1 , wherein the second digital model implements a convolutional neural network comprising:
a set of convolutional layers configured to generate feature matrices for the second input image; and
a fully-connected layer configured to determine, based on the feature matrices, whether the region of the second input image captures the object of the certain type associated with any of the plurality of classes.

6. The computer-implemented method of claim 5 ,
wherein the convolutional neural network further comprises a region-of-interest pooling layer configured to generate fixed-length representations of regions of the second input image; and
wherein output data from the region-of-interest pooling layer becomes input data to the fully-connected layer.

7. The computer-implemented method of claim 1 , further comprising, upon receiving the second set of training images, trimming each training image, in the second set of training images, to a fixed size.

8. The computer-implemented method of claim 1 , further comprising:
generating an additional set of training images using at least a subset of the second set of training images as input for a generative adversarial network; and
adding the additional set of training images to the augmented set of training images.

9. The computer-implemented method of claim 1 , wherein augmenting the second set of training images comprises:
creating an additional set of training images having copies of the second set of training images;
modifying at least a subset of the additional set of training images by performing at least one of: image rotation followed by image shearing, image scaling, and image enlargement followed by image chopping; and
generating, after the modifying, the augmented set of training images that include the additional set of training images.

10. The computer-implemented method of claim 1 , wherein identifying the plurality of regions from the third input image comprises:
generating a plurality of scaled input images from the third input image respectfully corresponding to the plurality of scales; and
identifying the plurality of regions, from the plurality of scaled input images, each likely to capture an object of the certain type.

11. The computer-implemented method of claim 1 , wherein the information indicating, for the candidate region, a position of the candidate region within the third input image and the certain type of the object.

12. A non-transitory computer-readable storage medium storing one or more instructions which, when executed by one or more processors, cause the one or more processors to perform a method of detecting infected objects from large field-of-view images, the method comprising:
receiving, by a processor, a first set of training images,
at least a certain image of the first set of training images capturing an object of a certain type in more than half of a total area of the certain image;

building, by the processor, a first digital model based on the first set of training images,
the first digital model configured to receive a list of feature vectors of a first input image and produce a determination of whether the first input image captures an object of the certain type;

receiving a second set of training images,
for each of a plurality of classes, at least a specific image of the second set of training images capturing an object of the certain type that is associated with the class in more than half of a total area of the specific image;

augmenting, for a plurality of scales, the second set of training images with copies of a group of the second set of training images resized at the plurality of scales to generate an augmented set of training images including the second set of training images;
creating a second digital model based on the augmented set of training images,
the second digital model configured to receive a second input image capturing a plurality of objects of the certain type and produce a determination of whether a region of the second input image captures an object of the certain type associated with any of the plurality of classes;

receiving a third input image capturing multiple objects of the certain type;
identifying a plurality of regions from the third input image using the first digital model;
computing a

Assignees

• Climate Corp

Inventors

• Chen Yaqi
• Guan Wei

Classifications

• G06V10/82

  using neural networks · CPC title

• G06Q50/02 Primary

  Agriculture; Fishing; Forestry; Mining · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2411

  based on the proximity to a decision surface, e.g. support vector machines · CPC title

• G06F18/24133

  Distances to prototypes · CPC title