Display control apparatus, display control method, and computer program product

Patent metadata

Field	Value
Publication number	US-12032625-B2
Application number	US-202117179709-A
Country	US
Kind code	B2
Filing date	Feb 19, 2021
Priority date	Apr 1, 2020
Publication date	Jul 9, 2024
Grant date	Jul 9, 2024

Abstract

According to an embodiment, a display control apparatus includes a clustering unit, a sub-clustering unit, and a display control unit. The clustering unit is configured to classify images into a plurality of clusters based on similarity degrees of the images and a first threshold. The sub-clustering unit is configured to further classify images within each of the plurality of clusters into a plurality of sub-clusters based on the similarity degrees and a second threshold that is higher than the first threshold. The display control unit is configured to display, on a display unit, display information including a cluster representative indicative of a representative of images included in a cluster and a sub-cluster representative indicative of a representative of an image included in a sub-cluster.

First claim

What is claimed is:

1. A display control apparatus comprising:
a memory; and
one or more hardware processors electrically coupled to the memory and configured to:
classify, based on images received, a similarity degree to an average of images included in a cluster, and a first threshold used for classification by the similarity degree, the images received into a plurality of clusters;
further classify images within each of the plurality of clusters into a plurality of sub-clusters based on the similarity degrees and a second threshold that is higher than the first threshold; and
display, on a display device, display information including a cluster representative indicative of a representative of images included in a cluster and a sub-cluster representative indicative of a representative of an image included in a sub-cluster, wherein
the cluster representative is an image that is closest to the average of the images included in the cluster among the images included in the cluster,
the sub-cluster representative is an image that is closest to an average of images included in the sub-cluster among the images included in the sub-cluster,
a plurality of images classified by the hardware processors has a tree structure in which a sub-cluster representative with a highest similarity degree to the cluster representative serves as a root, sub-cluster representatives are connected together in a first direction in descending order of similarity degrees to the cluster representative, and images in a first sub-cluster including a first sub-cluster representative are connected together in a second direction in descending order of similarity degrees to the first sub-cluster representative, and
the hardware processors are configured to, when classifying a new image, not calculate similarity degrees between the images in the first sub-cluster and the new image when a similarity degree between the new image and the first sub-cluster representative does not exceed the second threshold and proceed to calculate a similarity degree between the new image and any next sub-cluster representative in the first direction, and when there are no remaining next sub-cluster representatives in the first direction, create an end node that adds the new image to the tree structure.

2. The apparatus according to claim 1 , wherein the hardware processors are configured to display a predefined number of images included in the first sub-cluster in descending order of similarity degrees to the first sub-cluster representative.

3. The apparatus according to claim 1 , wherein
the images includes persons,
the hardware processors are configured to classify the images into the plurality of clusters for the respective persons based on the similarity degrees of the persons included in the images and the first threshold, and
the hardware processors are configured to further classify the images into the plurality of sub-clusters based on the similarity degrees of the persons included in the images and the second threshold.

4. The apparatus according to claim 3 , wherein the hardware processors are further configured to:
register, in a person search database, one or more images selected from the display information.

5. The apparatus according to claim 1 , wherein the hardware processors are further configured to:
delete one or more images selected from the display information or to change a cluster or a sub-cluster that includes one or more images selected from the display information.

6. A display control method comprising:
classifying, by a display control apparatus and based on images received, a similarity degree to an average of images included in a cluster, and a first threshold used for classification by the similarity degree, the images received into a plurality of clusters;
further classifying, by the display control apparatus, images within each of the plurality of clusters into a plurality of sub-clusters based on the similarity degrees and a second threshold that is higher than the first threshold; and
displaying, on a display device by the display control apparatus, display information including a cluster representative indicative of a representative of images included in a cluster and a sub-cluster representative indicative of a representative of an image included in a sub-cluster, wherein
the cluster representative is an image that is closest to the average of the images included in the cluster among the images included in the cluster,
the sub-cluster representative is an image that is closest to an average of images included in the sub-cluster among the images included in the sub-cluster,
a plurality of images classified at the further classifying into the plurality of sub-clusters has a tree structure in which a sub-cluster representative with a highest similarity degree to the cluster representative serves as a root, sub-cluster representatives are connected together in a first direction in descending order of similarity degrees to the cluster representative, and images in a first sub-cluster including a first sub-cluster are connected together in a second direction in descending order of similarity degrees to the first sub-cluster representative, and
when classifying a new image at the further classifying into the plurality of sub-clusters, similarity degrees between the images in the first sub-cluster and a new image are not calculated when a similarity degree between the new image and the first sub-cluster representative does not exceed the second threshold and a similarity degree is calculated between the new image and any next sub-cluster representative in the first direction, and when there are no remaining next sub-cluster representatives in the first direction, the method includes creating an end node that adds the new image to the tree structure.

7. The method according to claim 6 , wherein at the displaying, a predefined number of images included in the first sub-cluster is displayed in descending order of similarity degrees to the first sub-cluster representative.

8. The method according to claim 6 , wherein
the images includes persons,
at the classifying into the plurality of clusters, the images are classified into the plurality of clusters for the respective persons based on the similarity degrees of the persons included in the images and the first threshold, and
at the further classifying into the plurality of sub-clusters, the images are classified into the plurality of sub-clusters based on the similarity degrees of the persons included in the images and the second threshold.

9. The method according to claim 8 , further comprising registering, in a person search database, one or more images selected from the display information.

10. The method according to claim 6 , further comprising deleting one or more images selected from the display information or changing a cluster or a sub-cluster that includes one or more images selected from the display information.

11. A computer program product comprising a non-transitory computer-readable medium including programmed instructions, the instructions causing a computer to:
classify, based on images received, a similarity degree to an average of images included in a cluster, and a first threshold used for classification by the similarity degree, the images received into a plurality of clusters;
further classify images within each of the plurality of clusters into a plurality of sub-clusters based on the similarity degrees and a second threshold that is higher than the first threshold; and
display, on a display device, display information including a cluster representative indicative of a representative of images included in a cluster and a sub-clus

Assignees

• Toshiba Kk

Inventors

• Saito Hiroo
• Yamaji Yuto
• Yamaguchi Osamu

Classifications

• G06V10/7625

  Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms · CPC title

• G06V10/761

  Proximity, similarity or dissimilarity measures · CPC title

• G06F18/2431

  Multiple classes · CPC title

• G06F18/231

  Hierarchical techniques, i.e. dividing or merging pattern sets so as to obtain a dendrogram · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title