Methods and systems for low-energy image classification

Patent metadata

Field	Value
Publication number	US-2016267358-A1
Application number	US-201514715554-A
Country	US
Kind code	A1
Filing date	May 18, 2015
Priority date	Mar 11, 2015
Publication date	Sep 15, 2016
Grant date	—

Abstract

Examples of the disclosure enable efficient processing of images. In some examples, one or more interest points are identified in an image. One or more features are extracted from the identified interest points using a filter module, a gradient module, a pool module, and/or a normalizer module. The extracted features are aggregated to generate one or more vectors. Based on the generated vectors, it is determined whether the extracted features satisfy a predetermined threshold. Based on the determination, the image is classified such that the image is configured to be processed based on the classification. Aspects of the disclosure facilitate conserving memory at a local device, reducing processor load or an amount of energy consumed at the local device, and/or reducing network bandwidth usage between the local device and the remote device.

First claim

What is claimed is:

1 . A computer-implemented method for classifying one or more images, the method comprising executing on one or more computing devices the operations of:
identifying one or more interest points in an image; extracting one or more features from the identified interest points using one or more of a filter module, a gradient module, a pool module, and a normalizer module; aggregating the extracted features to generate one or more vectors; based on the generated vectors, determining whether the extracted features satisfy a predetermined threshold; and based on the determination, classifying the image such that the image is configured to be processed based on the classification including one or more of recognizing the extracted features, understanding the image, and generating one or more actionable items.

2 . The computer-implemented method of claim 1 , wherein identifying one or more interest points comprises detecting one or more corners in the image, wherein a first corner corresponds to a first interest point.

3 . The computer-implemented method of claim 1 , wherein extracting one or more features comprises smoothing, by the filter module, one or more pixels associated with the interest points.

4 . The computer-implemented method of claim 1 , wherein extracting one or more features comprises:
computing, by the gradient module, one or more gradients along a first axis and a second axis perpendicular to the first axis; and based on the computed gradients, generating, by the gradient module, an output array including one or more feature maps.

5 . The computer-implemented method of claim 4 , wherein generating an output array comprises generating the output array such that the output array includes a predetermined number of feature maps having a predetermined size.

6 . The computer-implemented method of claim 1 , wherein extracting one or more features comprises pooling, by the pool module, one or more feature maps along a grid, wherein the feature maps correspond to the extracted features.

7 . A mobile device comprising:
a sensor module configured to capture data corresponding to a plurality of images; a memory area storing computer-executable instructions for classifying the plurality of images; and a processor configured to execute the computer-executable instructions to:
extract one or more features from the plurality of images, a quantity of extracted features associated with a desired power consumption of the mobile device;
determine whether the extracted features satisfy a predetermined threshold; and
based on the determination, classify the plurality of images such that the plurality of images are configured to be processed based on the classification including one or more of recognize the extracted features, understand the images, and generate one or more actionable items.

8 . The mobile device of claim 7 , wherein the processor is further configured to execute the computer-executable instructions to detect one or more corners in the plurality of images, wherein the features are extracted from the detected corners.

9 . The mobile device of claim 7 , wherein the processor is further configured to execute the computer-executable instructions to smooth one or more pixels associated with the plurality of images, wherein the pixels correspond to the extracted features.

10 . The mobile device of claim 7 , wherein the processor is further configured to execute the computer-executable instructions to:
compute one or more gradients along a first axis and a second axis perpendicular to the first axis; and based on the computed gradients, generate an output array including one or more feature maps, such that the output array includes a predetermined number of feature maps having a predetermined size.

11 . The mobile device of claim 7 , wherein the processor is further configured to execute the computer-executable instructions to pool one or more feature maps along a grid, wherein the feature maps correspond to the extracted features.

12 . A computing device comprising:
a sensor module configured to capture data corresponding to one or more images; a feature computation module configured to:
identify one or more interest points in the images;
extract one or more features from the identified interest points; and
aggregate the extracted features to generate one or more vectors; and
a feature classification module configured to:
based on the generated vectors, determine whether the extracted features satisfy a predetermined threshold; and
based on the determination, classify the images into a first set of images and a second set of images; and
transmit the first set of images to a server, the server configured to process the first set of images including one or more of recognize the extracted features, understand the images, and generate one or more actionable items.

13 . The computing device of claim 12 , wherein the feature computation module is configured to detect one or more corners in the images, wherein a first corner corresponds to a first interest point.

14 . The computing device of claim 12 , wherein the feature computation module is configured to smooth one or more pixels associated with the interest points.

15 . The computing device of claim 12 , wherein the feature computation module is configured to:
compute one or more gradients along a first axis and a second axis perpendicular to the first axis; and based on the computed gradients, generate an output array including a predetermined number of feature maps.

16 . The computing device of claim 12 , wherein the feature computation module is configured to pool one or more feature maps along a grid, wherein the feature maps correspond to the extracted features.

17 . The computing device of claim 12 , wherein one or more of the feature computation module and the feature classification module are configured to process one image in parallel with processing another image.

18 . The computing device of claim 12 , wherein the feature computation module is configured to process an interest point in parallel with processing another interest point.

19 . The computing device of claim 12 , wherein the feature computation module is configured to process a pixel in parallel with processing another pixel.

20 . The computing device of claim 12 , wherein one or more of the feature computation module and the feature classification module includes a plurality of submodules, wherein a first submodule of the plurality of submodules is configured to process a first set of data to generate a first output and transmit the first output to a second submodule of the plurality of submodules such that the first submodule is configured to process a second set of data in parallel with the second submodule processing the first output.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Shoaib Mohammed
• Venkataramani Swagath

Classifications

• G06V10/82

  using neural networks · CPC title

• G06V10/809

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title