Neural Network Image Curation Control

Patent metadata

Field	Value
Publication number	US-2016179844-A1
Application number	US-201414573963-A
Country	US
Kind code	A1
Filing date	Dec 17, 2014
Priority date	Dec 17, 2014
Publication date	Jun 23, 2016
Grant date	—

Abstract

Neural network image curation techniques are described. In one or more implementations, curation is controlled of images that represent a repository of images. A plurality of images of the repository are curated by one or more computing devices to select representative images of the repository. The curation includes calculating a score based on image and face aesthetics, jointly, for each of the plurality of images through processing by a neural network, ranking the plurality of images based on respective said scores, and selecting one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository.

First claim

What is claimed is:

1 . A method to control curation of images that represent a repository of images, the method comprising:
curating a plurality of images of the repository by one or more computing devices to select representative images of the repository, the curating including:
calculating a score based on image and face aesthetics, jointly, for each of the plurality of images through processing by a neural network;
ranking the plurality of images based on respective said scores; and
selecting one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository.

2 . A method as described in claim 1 , wherein the calculating of the score based on image and face aesthetics, jointly, for each of the plurality of images includes:
generating a plurality of patches from the each said image; detecting activations of a plurality of image characteristics for each of the plurality of patches using the neural network, the plurality of image characteristics pertaining to image and face aesthetics; and the calculating of the score is based on at least in part on the detected activations.

3 . A method as described in claim 2 , wherein the activations describe an amount of a corresponding image characteristic.

4 . A method as described in claim 2 , wherein the calculating of the score includes aggregating the detected activations for the plurality of patches.

5 . A method as described in claim 2 , wherein the detected activations for a respective said patch are included in a vector that has a plurality of dimensions, each of the dimensions corresponding to a respective said image characteristic.

6 . A method as described in claim 1 , further comprising:
receiving the plurality of images responsive to an image search; and causing output of the representative images as an image search result of the image search.

7 . A method as described in claim 1 , further comprising controlling a representation of the repository in a user interface to include the representative images, the representation of the repository selectable via user interaction to cause output of the plurality of images in the user interface.

8 . A method as described in claim 1 , further comprising rejecting images that do not meet an image quality score threshold, face quality threshold, face aesthetic threshold, or image aesthetic threshold, and wherein the plurality of images processed as part of the curating meet the image quality score threshold.

9 . A method as described in claim 7 , wherein the image quality score threshold is based at least in part on sharpness, resolution, exposure, light. blur, and noise of respective said images.

10 . A method as described in claim 1 , wherein the image and face aesthetics are based at least in part on image characteristics including color and composition of respective said images.

11 . In a digital medium environment for curating images, where the images are included in a repository having a plurality of images, a method for representing the plurality of the image in the repository in the digital medium environment, the method comprising:
receiving a request to curate the plurality of images to select images that are representative of the plurality of images in the repository; for each said image, generating a plurality of patches from the image and detecting activations of a plurality of image characteristics for each of the plurality of patches using a neural network, the plurality of image characteristics pertaining to image and face aesthetics; and selecting one or more of the plurality of images using a greedy selection technique as one of the representative images of the repository based on the detected activations and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository.

12 . A method as described in claim 11 , wherein the activations describe an amount of a corresponding image characteristic.

13 . A method as described in claim 11 , wherein the detected activations for a respective said patch are included in a vector that has a plurality of dimensions, each of the dimensions corresponding to a respective said image characteristic.

14 . A method as described in claim 11 , further comprising rejecting images that do not meet an image quality score threshold and wherein the plurality of images processed as part of the curating meet the image quality score threshold.

15 . A method as described in claim 11 , wherein the receiving includes receiving the plurality of images as part of an image search result or the repository is a folder.

16 . A system to control curation of images to represent a repository of images, the system comprising:
an image curation module implemented at least partially in hardware to curate a plurality of images of the repository to select representative images of the repository, the image curation module including:
a joint image and face aesthetic calculation module to calculate a score based on image and face aesthetics, jointly, for each of the plurality of images through processing by a neural network;
an image ranking module to rank the plurality of images based on respective said scores; and
an image selection module to select one or more of the plurality of images as one of the representative images of the repository based on the ranking and a determination that the one or more said images are not visually similar to images that have already been selected as one of the representative images of the repository.

17 . A system as described in claim 16 , wherein the image curation module further comprises an image quality score module configured to reject images that do not meet an image quality score threshold.

18 . A system as described in claim 16 , wherein joint image and face aesthetic calculation module is configured to calculate the score based on image and face aesthetics, jointly, for each of the plurality of images by:
generating a plurality of patches from the each said image; detecting activations of a plurality of image characteristics for each of the plurality of patches using the neural network, the plurality of image characteristics pertaining to image and face aesthetics; and the calculating of the score is based on at least in part on the detected activations.

19 . A system as described in claim 18 , wherein the calculating of the score includes aggregating the detected activations for the plurality of patches.

20 . A system as described in claim 18 , wherein the detected activations for a respective said patch are included in a vector that has a plurality of dimensions, each of the dimensions corresponding to a respective said image characteristic.

Assignees

• Adobe Systems Inc

Inventors

• Shen Xiaohui
• Lu Xin
• Lin Zhe
• Mech Radomir

Classifications

• G06V30/19173

  Classification techniques · CPC title

• G06F16/5838

  using colour · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F18/24137

  Distances to cluster centroïds · CPC title