Information processing apparatus, information processing method, and non-transitory computer readable medium

Patent metadata

Field	Value
Publication number	US-12548289-B2
Application number	US-202117915857-A
Country	US
Kind code	B2
Filing date	Oct 8, 2021
Priority date	Oct 8, 2021
Publication date	Feb 10, 2026
Grant date	Feb 10, 2026

Abstract

An information processing apparatus ( 1 ) includes: an acquisition unit ( 101 ) configured to acquire an object image that contains one or more objects; a first inference unit ( 105 ) configured to infer one or more rectangular areas that respectively surround the one or more objects in the object image, and a type of each of the one or more objects, by applying the object image to a first learning model; a second inference unit ( 105 ) configured to infer an area of a target object in the object image by applying the one or more rectangular areas and the type of each of the one or more rectangular areas to a second learning model; and an extraction unit ( 105 ) configured to perform color determination on each pixel in the area of the target object and extract colors of the target object.

First claim

The invention claimed is:

1 . An information processing apparatus comprising:
at least one memory configured to store program code; at least one processor configured to access the at least one memory and operate as instructed by the program code, the program code including: acquisition code configured to cause at least one of the at least one processor to acquire an object image that contains one or more objects; first inference code configured to cause at least one of the at least one processor to infer one or more rectangular areas that respectively surround the one or more objects in the object image, and a type of each of the one or more objects, by applying the object image to a first learning model; second inference code configured to cause at least one of the at least one processor to infer an area of a target object in the object image by applying the one or more rectangular areas and the type of each of the one or more objects to a second learning model; extraction code configured to cause at least one of the at least one processor to extract colors of the target object; generation code configured to cause at least one of the at least one processor to generate a plurality of feature vectors for the target object by applying a color feature vector indicating the colors of the target object extracted by the extraction unit, and the object image, to a plurality of learning models; and search code configured to cause at least one of the at least one processor to search for similar images that are similar to the object image of the target object, using the color feature vector and the plurality of feature vectors.

2 . The information processing apparatus according to claim 1 ,
wherein the extraction code is configured to cause at least one of the at least one processor to perform color determination on each pixel in the area of the target object and extracts colors of the target object.

3 . The information processing apparatus according to claim 2 ,
wherein the extraction code is configured to cause at least one of the at least one processor to perform color determination on each pixel in the area of the target object, and extract a predetermined number of top colors from among one or more colors determined in the area of the target object, as the colors of the target object.

4 . The information processing apparatus according to claim 1 ,
wherein the first learning model is a YOLO (You Only Look Once) learning model.

5 . The information processing apparatus according to claim 1 ,
wherein the second learning model is a learning model constituted by an FCN (Fully Convolutional Network).

6 . The information processing apparatus according to claim 1 ,
wherein the object image is image data generated from a Y element, a Cb element, and a Cr element of data generated from a YCbCr image through a DCT conversion.

7 . The information processing apparatus according to claim 6 ,
wherein the image data is data generated by adjusting sizes of Y elements, Cb elements, and Cr elements of pieces of data subjected to DCT conversion, and concatenating the pieces of data with each other.

8 . The information processing apparatus according to claim 1 , further comprising:
concatenation code configured to cause at least one of the at least one processor to concatenate and embed the color feature vector and the plurality of feature vectors into a common feature space to generate a compounded feature vector in the feature space, wherein the search code is configured to cause at least one of the at least one processor to search for similar images that are similar to the object image of the target object, using the compounded feature vector.

9 . An information processing method performed by at least one processor and comprising:
acquiring an object image that contains one or more objects; inferring one or more rectangular areas that respectively surround the one or more objects in the object image, and a type of each of the one or more objects, by applying the object image to a first learning model; inferring an area of a target object in the object image by applying the one or more rectangular areas and the type of each of the one or more objects to a second learning model; extracting colors of the target object; generating a plurality of feature vectors for the target object by applying a color feature vector indicating the colors of the target object extracted by the extraction unit, and the object image, to a plurality of learning models; and searching for similar images that are similar to the object image of the target object, using the color feature vector and the plurality of feature vectors.

10 . A non-transitory computer readable medium storing a computer program for causing a computer to:
acquire an object image that contains one or more objects; infer one or more rectangular areas that respectively surround the one or more objects in the object image, and a type of each of the one or more objects, by applying the object image to a first learning model; acquire an area of a target object in the object image by applying the one or more rectangular areas and the type of each of the one or more rectangular areas to a second learning model; extract colors of the target object; generate a plurality of feature vectors for the target object by applying a color feature vector indicating the colors of the target object extracted by the extraction unit, and the object image, to a plurality of learning models; and search for similar images that are similar to the object image of the target object, using the color feature vector and the plurality of feature vectors.

Assignees

• Rakuten Group Inc

Inventors

• Arslan Hasan
• CUEVAS JUAREZ Josue
• SANAGAVARAPU Rajasekhar

Classifications

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T2207/20052

  Discrete cosine transform [DCT] · CPC title

• G06T2207/10024

  Color image · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title