Color-based post-processing of images

Patent metadata

Field	Value
Publication number	US-9805289-B2
Application number	US-201514975639-A
Country	US
Kind code	B2
Filing date	Dec 18, 2015
Priority date	Dec 18, 2015
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

The disclosure includes a system and method for distinguishing objects of similar appearance that vary in color. An image recognition application receives a query image of a plurality of objects, determines a candidate list of indexed images that match an object in the query image based on luminance features, crops an image of the object from the query image to generate a cropped image, computes color features for the object, ranks the candidate list of indexed images based on the color features, and provides the candidate list of indexed images to a user.

First claim

What is claimed is:

1. A method comprising:
receiving, by one or more processors, a query image of a plurality of objects;
determining, by the one or more processors, a candidate list of indexed images that match an object in the query image based on luminance features;
cropping, by the one or more processors, an image of the object from the query image to generate a cropped image;
computing, by the one or more processors, a color transfer function using sample colors collected from the cropped image and an indexed image of the candidate list;
ranking, by the one or more processors, the candidate list of indexed images based on one or more distance measures computed using the color transfer function; and
providing, by the one or more processors, the ranked candidate list of indexed images to a user.

2. The method of claim 1 , further comprising:
determining, by the one or more processors, a region of interest for the object in the query image based on the luminance features; and
determining, by the one or more processors, a region of interest for the indexed image based on the luminance features.

3. The method of claim 2 , wherein computing the color transfer function further comprises collecting the sample colors from the region of interest determined for the cropped image and the region of interest determined for the indexed image.

4. The method of claim 3 , wherein computing the color transfer function further comprises:
applying, by the one or more processors, the color transfer function to the cropped image to generate a transformed cropped image;
applying, by the one or more processors, the color transfer function to the indexed image to generate a transformed indexed image;
computing, by the one or more processors, a first distance measure between the cropped image and the transformed indexed image; and
computing, by the one or more processors, a second distance measure between the indexed image and the transformed cropped image.

5. The method of claim 4 , wherein ranking the candidate list of indexed images is based on the first distance measure and the second distance measure.

6. The method of claim 4 , wherein the first distance and the second distance are an earth mover's distance.

7. The method of claim 1 , further comprising filtering the candidate list of indexed images based on the ranking.

8. A system comprising;
one or more processors; and
a memory, the memory storing instructions, which when executed cause the one or more processor to:
receive a query image of a plurality of objects;
determine a candidate list of indexed images that match an object in the query image based on luminance features;
crop an image of the object from the query image to generate a cropped image;
compute a color transfer function using sample colors collected from the cropped image and an indexed image of the candidate list;
rank the candidate list of indexed images based on one or more distance measures computed using the color transfer function; and
provide the ranked candidate list of indexed images to a user.

9. The system of claim 8 , wherein the instructions further cause the one or more processors to:
determine a region of interest for the object in the query image based on the luminance features; and
determine a region of interest for the indexed image based on the luminance features.

10. The system of claim 9 , wherein to compute the color transfer function, the instructions cause the one or more processors to collect the sample colors from the region of interest determined for the cropped image and the region of interest determined for the indexed image.

11. The system of claim 10 , wherein to compute the color transfer function, the instructions cause the one or more processors to:
apply the color transfer function to the cropped image to generate a transformed cropped image;
apply the color transfer function to the indexed image to generate a transformed indexed image;
compute a first distance measure between the cropped image and the transformed indexed image; and
compute a second distance measure between the indexed image and the transformed cropped image.

12. The system of claim 11 , wherein the instructions cause the one or more processors to rank the candidate list of indexed images based on the first distance measure and the second distance measure.

13. The system of claim 11 , wherein the first distance measure and the second distance measure are a Pearson correlation coefficient.

14. The system of claim 8 , wherein the instructions further cause the one or more processors to filter the candidate list of indexed images based on the ranking.

15. A computer program product comprising a non-transitory computer readable medium storing a computer readable program, wherein the computer readable program when executed causes a computer to perform operations comprising:
receiving a query image of a plurality of objects;
determining a candidate list of indexed images that match an object in the query image based on luminance features;
cropping an image of the object from the query image to generate a cropped image;
computing a color transfer function using sample colors collected from the cropped image and an indexed image of the candidate list;
ranking the candidate list of indexed images based on one or more distance measures computed using the color transfer function; and
providing the ranked candidate list of indexed images to a user.

16. The computer program product of claim 15 , wherein the computer readable program further causes the computer to:
determine a region of interest for the object in the query image based on the luminance features; and
determine a region of interest for the indexed image based on the luminance features.

17. The computer program product of claim 16 , wherein computing the color transfer function further comprises collecting the sample colors from the region of interest determined for the cropped image and the region of interest determined for the indexed image.

18. The computer program product of claim 17 , wherein computing the color transfer function further comprises:
applying the color transfer function to the cropped image to generate a transformed cropped image;
applying the color transfer function to the indexed image to generate a transformed indexed image;
computing a first distance measure between the cropped image and the transformed indexed image; and
computing a second distance measure between the indexed image and the transformed cropped image.

19. The computer program product of claim 18 , wherein ranking the candidate list of indexed images is based on the first distance measure and the second distance measure.

20. The computer program product of claim 15 , wherein the computer readable program further causes the computer to filter the candidate list of indexed images based on the ranking.

Assignees

• Narasimha Ramya
• Moraleda Jorge
• Berkner Kathrin
• Bhat Srikrishna
• Pavani Sri Kaushik
• Garg Anshul
• Ricoh Co Ltd

Inventors

• Narasimha Ramya
• Moraleda Jorge
• Berkner Kathrin
• Bhat Srikrishna
• Pavani Sri Kaushik
• Garg Anshul

Classifications

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06V10/56 Primary

  relating to colour · CPC title

• G06T7/408

  Physics · mapped topic

• H04N9/7973

  by dividing the luminance or colour component signal samples or frequency bands among a plurality of recording channels (H04N9/804, H04N9/825 take precedence) · CPC title

• G06K9/4652

  Physics · mapped topic