Electronic device and control method thereof

Patent metadata

Field	Value
Publication number	US-11816149-B2
Application number	US-202117171117-A
Country	US
Kind code	B2
Filing date	Feb 9, 2021
Priority date	Feb 11, 2020
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

An electronic device and a method for controlling thereof are provided. A method for controlling an electronic device according to the disclosure includes obtaining a plurality of images for performing clustering, obtaining a plurality of target areas corresponding to each of the plurality of images, obtaining a plurality of feature vectors corresponding to the plurality of target areas, obtaining a plurality of central nodes corresponding to the plurality of feature vectors, obtaining neighbor nodes associated with each of the plurality of central nodes, obtaining a subgraph based on the plurality of central nodes and the neighbor nodes, identifying the connection probabilities between the plurality of central nodes of the subgraph and the neighbor nodes of each of the plurality of central nodes based on a graph convolutional network, and clustering the plurality of target areas based on the identified connection probabilities.

First claim

What is claimed is:

1. A method for controlling an electronic device, the method comprising:
obtaining a plurality of images for performing clustering;
obtaining a plurality of target areas, wherein each of the target areas corresponds to an image of the plurality of images;
obtaining a plurality of feature vectors corresponding to the plurality of target areas;
obtaining a plurality of central nodes corresponding to the plurality of feature vectors;
obtaining neighbor nodes, wherein each of the neighbor nodes is associated with a central node of the plurality of central nodes;
obtaining a subgraph based on the plurality of central nodes and the neighbor nodes;
identifying connection probabilities between the plurality of central nodes of the subgraph and the neighbor nodes of each of the plurality of central nodes based on a graph convolutional network; and
clustering the plurality of target areas based on the connection probabilities.

2. The method of claim 1 , wherein the obtaining of the subgraph comprises:
identifying one of the plurality of feature vectors as corresponding to the central node;
obtaining the neighbor node associated with the central node based on feature vectors different from the feature vector corresponding to the central node; and
constructing the subgraph according to the central node and the neighbor node.

3. The method of claim 2 , wherein the obtaining of the neighbor node comprises:
obtaining cosine distances between the feature vector corresponding to the central node and the feature vectors different from the feature vector corresponding to the central node; and
screening the neighbor node from the feature vectors different from the feature vector corresponding to the central node based on the cosine distances.

4. The method of claim 3 , further comprising using a hierarchical clustering module to maintain a difference between noise and other node features.

5. The method of claim 3 , wherein the obtaining of the cosine distances comprises:
using each node as a central node, selecting first-order neighbor nodes according to the cosine distance;
selecting neighbor nodes of the first-order neighbor nodes as second-order neighbor nodes according to the cosine distance; and
selecting K neighbor nodes for each node to construct the subgraph.

6. The method of claim 3 , further comprising determining the connection probability of a central node and the central node's first-order neighbor nodes according to a softmax function.

7. The method of claim 6 , further comprising repeating the method until the subgraph of each node is established.

8. The method of claim 1 , wherein the obtaining of the connection probabilities comprises:
obtaining node embedding of the subgraph according to at least one layer of the graph convolutional network; and
obtaining a connection probability between the central node of the subgraph and a neighbor node thereof based on the node embedding.

9. The method of claim 8 , wherein the obtaining of the node embedding comprises:
obtaining input features corresponding to the subgraph; and
inputting the input features into the at least one layer of the graph convolutional network for feature extraction and using an obtained output result as the node embedding.

10. The method of claim 1 , wherein the clustering of the plurality of target areas according to the connection probabilities comprises:
obtaining, for at least one target area to be clustered, a set of neighbor nodes in a same category according to the connection probabilities;
adjusting, for at least one neighbor node in the set of neighbor nodes in the same category, a connection probability of the at least one neighbor node and the central node according to the connection probabilities of the at least one neighbor node and respective neighbor nodes in the set of neighbor nodes in the same category;
clustering the plurality of target areas based on the adjusted connection probability to determine whether the at least one neighbor node is a neighbor node in the same category; and
clustering respective target areas to be clustered according to the neighbor nodes in the same category corresponding to the respective target areas to be clustered.

11. The method of claim 10 , wherein the obtaining a set of neighbor nodes comprises:
determine, for the at least one target area to be clustered, neighbor nodes of which connection probabilities with the central node are not less than a preset threshold, and forming the determined neighbor nodes as the set of neighbor nodes in the same category.

12. The method of claim 10 , wherein the adjusting of the connection probability comprises:
determining, for at least one neighbor node in the set of neighbor nodes in the same category, an average connection probability of the at least one neighbor node and respective neighbor nodes in the set of neighbor nodes in a same category as the connection probability of the at least one neighbor node and the central node.

13. The method of claim 10 , wherein the clustering of the respective target areas comprises:
determining connection probabilities among central nodes corresponding to respective target areas to be clustered according to neighbor nodes in the same category corresponding to the respective target areas to be clustered; and
clustering the respective target areas to be clustered based on the connection probabilities among the respective central nodes.

14. The method of claim 1 , further comprising:
receiving a keyword for image search input by a user;
identifying a category associated with the keyword based on a clustering result; and
searching for images matching the keyword among images in the identified category.

15. An electronic device comprising:
a memory storing at least one instruction; and
a processor that is connected to the memory and controls the electronic device,
wherein the processor is configured to, by executing the at least one instruction:
obtain a plurality of images for performing clustering,
obtain a plurality of target areas, wherein each target area corresponds to an image of the plurality of images,
obtain a plurality of feature vectors corresponding to the plurality of target areas,
obtain a plurality of central nodes corresponding to the plurality of feature vectors,
obtain neighbor nodes, wherein each neighbor node is associated with a central node of the plurality of central nodes,
obtain a subgraph based on the plurality of central nodes and the neighbor nodes,
identify connection probabilities between the plurality of central nodes of the subgraph and the neighbor nodes of each of the plurality of central nodes based on a graph convolutional network, and
cluster the plurality of target areas based on the connection probabilities.

16. The electronic device of claim 15 , wherein the processor is further configured to:
identify one of the plurality of feature vectors as corresponding to the central node,
obtain the neighbor node associated with the central node based on feature vectors different from the feature vector corresponding to the central node, and
construct the subgraph according to the central node and the neighbor node.

17. The electronic device of claim 16 , wherein the processor is further configured to:
obtain cosine distances between the feature vector corresponding to the central node and the feature vectors different from the feature vector corresponding to the central node, and
screen the neighbor node from the feature vectors different from the feature vector corres

Assignees

• Samsung Electronics Co Ltd

Inventors

• Zhang Dafeng
• Li Yanchao
• Gao Yanjun
• Wang Lei
• Jin Zhezhu
• Jiang Yingying
• Zhang Lanlan

Classifications

• G06F16/583 Primary

  using metadata automatically derived from the content · CPC title

• G06F16/5854

  using shape and object relationship · CPC title

• G06F18/251

  of input or preprocessed data · CPC title

• G06V10/7625

  Hierarchical techniques, i.e. dividing or merging patterns to obtain a tree-like representation; Dendograms · CPC title

• G06V10/7635

  based on graphs, e.g. graph cuts or spectral clustering · CPC title