Determining colors of objects in digital images

Patent metadata

Field	Value
Publication number	US-12118752-B2
Application number	US-202217658799-A
Country	US
Kind code	B2
Filing date	Apr 11, 2022
Priority date	Jul 22, 2019
Publication date	Oct 15, 2024
Grant date	Oct 15, 2024

Abstract

The present disclosure relates to a color classification system that accurately classifies objects in digital images based on color. In particular, in one or more embodiments, the color classification system utilizes a multidimensional color space and one or more color mappings to match objects to colors. Indeed, the color classification system can accurately and efficiently detect the color of an object utilizing one or more color similarity regions generated in the multidimensional color space.

First claim

What is claimed is:

1. A method comprising:
receiving a search query that comprises an indication of a target color and a corresponding query object; and
identifying the corresponding query object in a digital image based on the indication of the target color by:
identifying an object from a plurality of objects in the digital image, the object being formed by a plurality of pixels;
mapping pixels of the plurality of pixels to a multidimensional color space to determine a set of color correspondences to a set of color similarity regions of the multidimensional color space, each of the color similarity regions being associated with a color and one or more alternative versions of the color;
generating a set of color-matching scores for the object based on the set of color correspondences between the plurality of pixels and the set of color similarity regions;
classifying the object as a first color associated with a first color similarity region based on the set of color-matching scores; and
determining, based on a match between the indication of the target color and the first color, that the object is the corresponding query object.

2. The method of claim 1 , further comprising:
receiving a search request for the object having the first color among a dataset of digital images;
detecting a plurality of digital images that include the object from the dataset of digital images, the plurality of digital images including the digital image;
generating a color-matching score for the object detected within each of the plurality of digital images with respect to the first color;
identifying a subset of digital images of the plurality of digital images that comprise the object with a color-matching score satisfying a minimum color-matching threshold for the first color, the subset of digital images including the digital image; and
provide the subset of digital images in response to the search request.

3. The method of claim 1 , further comprising:
detecting the plurality of objects within the digital image, wherein the plurality of objects comprise a plurality of instances of the object;
generating a color-matching score for each instance of the object;
classifying one or more instances of the object as having the first color based on the set of color matching scores; and
returning the one or more instances of the object as having the first color.

4. The method of claim 1 , further comprising:
receiving the search query request for the object that comprises the indication of the target color where the target color is further indicated by a color name;
identifying, in response to the search query indicating the color name of the target color, the first color similarity region for the first color within the multidimensional color space that is pre-mapped to a first color name; and
returning, in response to identifying the first color similarity region for the first color within the multidimensional color space, an indication that the object in the digital image matches the first color based on the object being classified as the first color.

5. The method of claim 1 , further comprising identifying the object in the digital image utilizing an object detection neural network.

6. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:
receiving a search query that comprises an indication of a target color and a corresponding query object; and
identifying the corresponding query object in a digital image based on the indication of the target color by:
identifying an object from a plurality of objects in the digital image, the object being formed by a plurality of pixels;
mapping pixels of the plurality of pixels to a multidimensional color space to determine a set of color correspondences to a set of color similarity regions of the multidimensional color space, each of the color similarity regions being associated with a color and one or more alternative versions of the color;
generating a set of color-matching scores for the object based on the set of color correspondences between the plurality of pixels and the set of color similarity regions;
classifying the object as a first color associated with a first color similarity region based on the set of color-matching scores; and
determining, based on a match between the indication of the target color and the first color, that the object is the corresponding query object.

7. The non-transitory computer-readable medium of claim 6 , further comprising instructions that, when executed by the at least one processor, the at least one processor to perform operations comprising: generating a second color similarity region corresponding to a second color by grouping one or more additional alternative versions of the second color and the second color together within the multidimensional color space.

8. The non-transitory computer-readable medium of claim 6 , further comprising instructions that, when executed by the at least one processor, the at least one processor to perform operations comprising: determining an alternative version of the first color by:
converting the first color from a first color model corresponding to the multidimensional color space to a second color model;
modifying one or more color attributes of the first color within the second color model; and
converting the first color with the modified one or more color attributes from the second color model back to the first color model corresponding to the multidimensional color space.

9. The non-transitory computer-readable medium of claim 8 , wherein the one or more color attributes of the first color modified within the second color model comprise a color brightness, a color hue, or a color saturation level.

10. The non-transitory computer-readable medium of claim 8 , wherein modifying the one or more color attributes of the first color within the second color model comprises reducing a brightness level of a first color copy in the second color model.

11. The non-transitory computer-readable medium of claim 6 , wherein the first color similarity region for the first color comprises a first color point for the first color and a plurality of mapped alternative color points for the first color mapped to the multidimensional color space.

12. The non-transitory computer-readable medium of claim 11 , wherein generating the set of color-matching scores for the object based on the set of color correspondences between the plurality of pixels and the set of color similarity regions comprises generating a first color-matching score for the object based on determining distances in the multidimensional color space between each of the pixels of the plurality of pixels of the object and each of the plurality of mapped alternative color points and the first color point.

13. The non-transitory computer-readable medium of claim 12 , wherein classifying the object as the first color associated with the first color similarity region based on the set of color-matching scores comprises:
assigning a pixel as valid based on the pixel being within a minimum threshold distance to at least one of the plurality of mapped alternative color points corresponding to the first color or to the first color point; and
determining that the object matches the first color based on identifying a minimum percentage of pixels of the object being assigned as valid.

14. A system comprising:
one or more memory devices comprising a multidimensional color space for a plurality of colors comprising a plurality of color similarity regio

Assignees

• Adobe Inc

Inventors

• Ding Zhihong
• Cohen Scott
• Lin Zhe
• Ling Mingyang

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V10/56

  relating to colour · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title