Classifying colors of objects in digital images

Patent metadata

Field	Value
Publication number	US-11302033-B2
Application number	US-201916518795-A
Country	US
Kind code	B2
Filing date	Jul 22, 2019
Priority date	Jul 22, 2019
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

The present disclosure relates to a color classification system that accurately classifies objects in digital images based on color. In particular, in one or more embodiments, the color classification system utilizes a multidimensional color space and one or more color mappings to match objects to colors. Indeed, the color classification system can accurately and efficiently detect the color of an object utilizing one or more color similarity regions generated in the multidimensional color space.

First claim

What is claimed is:

1. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
generate a multidimensional color space for a plurality of colors comprising a plurality of color similarity regions each associated with a color name, the multidimensional color space comprising a first color similarity region associated with a first color name that groups regions of one or more alternative versions of a first color and a region of the first color within the multidimensional color space;
identify an object in a digital image comprising a plurality of pixels;
map pixels of the plurality of pixels to the multidimensional color space to determine one or more color correspondences to one or more color similarity regions of the plurality of color similarity regions and corresponding to one or more color names;
generate one or more color-matching scores for the object based on the one or more color correspondences between the pixels of the plurality of pixels and the one or more color similarity regions; and
classify the object as the first color having a first color name of the plurality of colors based on the one or more color-matching scores.

2. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to downsample the plurality of pixels before generating the one or more color-matching scores for the object.

3. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate a second color similarity region corresponding to a second color by grouping one or more additional alternative versions of the second color and the second color together within the multidimensional color space.

4. The non-transitory computer-readable medium of claim 3 , further comprising instructions that, when executed by the at least one processor, cause the computing device to generate an alternative version of the first color by:
converting the first color from a first color model corresponding to the multidimensional color space to a second color model;
modifying one or more color attributes of the first color within the second color model; and
converting the first color with the modified one or more color attributes from the second color model back to the first color model corresponding to the multidimensional color space.

5. The non-transitory computer-readable medium of claim 4 , wherein the one or more color attributes of the first color modified within the second color model comprise a color brightness, a color hue, or a color saturation level.

6. The non-transitory computer-readable medium of claim 4 , wherein the instructions, when executed by the at least one processor, cause the computing device to modify the one or more color attributes of the first color within the second color model by reducing a brightness level of a first color copy in the second color model.

7. The non-transitory computer-readable medium of claim 3 , wherein a first color similarity region for the first color comprises a first color point for the first color and a plurality of mapped alternative color points for the first color mapped to the multidimensional color space.

8. The non-transitory computer-readable medium of claim 7 , wherein the instructions, when executed by the at least one processor, cause the computing device to generate a first color-matching score for the object based on determining distances in the multidimensional color space between each of the pixels of the plurality of pixels of the object and each of the plurality of alternative mapped color points and the first color point.

9. The non-transitory computer-readable medium of claim 7 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
assign a pixel as valid based on the pixel being within a minimum threshold distance to at least one of the plurality of mapped color points corresponding to the first color or to the first color point; and
determine that the object matches the first color based on identifying a minimum percentage of pixels of the object being assigned as valid.

10. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
receive a search request for the object having the first color;
detect a plurality of digital images that include the object;
generate a color-matching score for the object detected within each of the plurality of digital images with respect to the first color;
identify a subset of digital images of the plurality of digital images that comprise the object with a color-matching score satisfying a minimum color-matching threshold for the first color; and
provide the subset of digital images in response to the search request.

11. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
detect a plurality of instances of the object within the digital image;
generate a color-matching score for each instance of the object;
classify one or more instances of the object to the first color based on the color matching scores; and
return the one or more instances of the object as having the first color.

12. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to:
receive a query request for the object indicating the first color by a color name;
identify, in response to the query request indicating the color name of the first color, a color similarity region for a first color within the multidimensional color space that is pre-mapped to a first color name; and
return, in response to identifying the color similarity region for a first color within the multidimensional color space, an indication that object in the digital image matches the first color based on the object being classified as the first color.

13. The non-transitory computer-readable medium of claim 1 , further comprising instructions that, when executed by the at least one processor, cause the computing device to identify the object in the digital image utilizing an object detection neural network.

14. A system for classifying objects in digital images based on color, the system comprising:
one or more memory devices comprising a multidimensional color space for a plurality of colors comprising a plurality of color similarity regions each associated with a color label, the multidimensional color space comprising a first color similarity region associated with a first color label that groups regions of one or more alternative versions of a first color and a region of the first color within the multidimensional color space; and
at least one server device configured to cause the system to:
identify an object in a digital image comprising a plurality of pixels;
map pixels of the plurality of pixels to the multidimensional color space to determine correspondences to one or more color similarity regions of the plurality of color similarity regions and corresponding to one or more color labels;
generate one or more color-matching scores for the object based on the one or more color correspondences between the pixels of the plurality of pixels and the one or more color similarity regions; and
classify the object

Assignees

• Adobe Inc

Inventors

• Ding Zhihong
• Cohen Scott
• Lin Zhe
• Ling Mingyang

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/56

  relating to colour · CPC title

• G06F18/213

  Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods · CPC title

• G06F18/241 Primary

  relating to the classification model, e.g. parametric or non-parametric approaches · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title