Type prediction method, apparatus and electronic device for recognizing an object in an image

Patent metadata

Field	Value
Publication number	US-10706334-B2
Application number	US-201815900572-A
Country	US
Kind code	B2
Filing date	Feb 20, 2018
Priority date	Feb 20, 2017
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

Type prediction method, apparatus and electronic device for recognizing an object in an image are disclosed. The method may include processing an image to be processed using a full image recognition technique to obtain a first type prediction result of an object in the image to be processed; processing a subject area of the image to be processed using a feature recognition technique to obtain a second type prediction result of an object of the subject area; determining whether the first type prediction result matches the second type prediction result; if the first type prediction result matches the second type prediction result, determining a type of the object of the image to be processed to be the first type prediction result or the second type prediction result.

First claim

What is claimed is:

1. A method comprising:
processing an image using full image recognition to obtain a first type prediction result of an object in the image based on full image feature data of the image;
processing a subject area of the image using feature recognition to obtain a second type prediction result of an object in the subject area in the image based on feature data of the subject area;
determining whether the first type prediction result matches the second type prediction result;
upon determining that the first type prediction result matches the second type prediction result, determining a type of the object in the image to be either the first type prediction result or the second type prediction result; and
upon determining that that the first type prediction result does not match the second type prediction result:
processing the subject area of the image using the full image recognition technique associated,
obtaining a third type of prediction result of the object in the subject area, and
determining the type of the object of the image to be processed to be the third type of prediction result or the second type prediction result.

2. The method of claim 1 , wherein processing the image using the full image recognition to obtain the first type prediction result of the object in the image comprises:
calculating visual feature data of the full image feature data of the image;
performing a classification prediction based on the visual feature data; and
obtaining the first type prediction result of the object in the image.

3. The method of claim 1 , wherein processing the subject area of the image using the feature recognition to obtain the second type prediction result of the object of the subject area comprises:
detecting a subject in the image to determine the subject area containing the subject;
calculating subject feature data of the subject area;
performing a classification prediction based on the subject feature data; and
obtaining the second type prediction result corresponding to the subject area.

4. The method of claim 3 , wherein: if the image includes a plurality of subjects, detecting the subject area including the subject comprises:
determining candidate areas containing at least one subject in the image;
selecting a candidate area meeting a predetermined condition from the candidate areas as the subject area of the image.

5. The method of claim 4 , wherein a candidate area is determined by:
analyzing the image using a selected subject analysis technique to recognize a subject of the image;
determining region scope for the subject based on coordinate information of boundary pixels of the subject; and
treating the region scope as the candidate area.

6. The method of claim 1 , wherein determining whether the first type prediction result matches the second type prediction result comprises:
determining whether label data of the first type prediction result is identical to label data of the second type prediction result, and determining that the first type prediction result matches the second type prediction result if the label data of the first type prediction result is identical to label data of the second type prediction result; or
determining whether the label data of the first type prediction result and the label data of the second type prediction result belong to a same classification prediction, and determining that the first type prediction result matches the second type prediction result if the label data of the first type prediction result and the label data of the second type prediction result belong to the same classification prediction.

7. The method of claim 1 , further comprising:
determining that the image includes a plurality of subjects by the full image recognition;
determining respective subject areas of subjects; and
obtaining respective second type prediction results corresponding to the respective subject areas.

8. The method of claim 7 , wherein determining whether the first type prediction result matches the second type prediction result comprises:
determining whether a second type prediction result in the respective second type prediction results matching the first type prediction result.

9. The method of claim 8 , wherein determining that the first type prediction result matches the second type prediction result if a second type prediction result in the respective second type prediction results matching the first type prediction result, and wherein the method further comprises:
designating the second type prediction result matching the first type prediction result as the type of the object of the image.

10. One or more computer readable media storing executable instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:
processing an image using a full image recognition technique to obtain a first type prediction result of an object in the image;
processing a subject area of the image using a feature recognition technique to obtain a second type prediction result of the object in the subject area;
determining whether the first type prediction result matches the second type prediction result;
upon determining that the first type prediction result matches the second type prediction result, determining a type of the object in the image to be the first type prediction result or the second type prediction result; and
upon determining that that the first type prediction result does not match the second type prediction result:
processing the subject area of the image using the full image recognition technique associated,
obtaining a third type of prediction result of the object in the subject area, and
determining the type of the object of the image to be processed to be the third type of prediction result or the second type prediction result.

11. The one or more computer readable media of claim 10 , wherein processing the image using the full image recognition technique to obtain the first type prediction result of the object in the image comprises:
calculating visual feature data of the full image feature data of the image;
performing a classification prediction based on the visual feature data; and
obtaining the first type prediction result of the object in the image.

12. The one or more computer readable media of claim 10 , wherein processing the subject area of the image using the feature recognition technique to obtain the second type prediction result of the object of the subject area comprises
analyzing a subject in the image to determine the subject area including the subject;
calculating subject feature data of the subject area;
performing a classification prediction based on the subject feature data; and
obtaining the second type prediction result corresponding to the subject area.

13. The one or more computer readable media of claim 12 , wherein: if the image includes a plurality of subjects, determining the subject area including the subject comprises selecting a candidate area meeting a predetermined condition from candidate areas having the subjects as the subject area of the image, and wherein the candidate area is an image area having a subject from the image.

14. The one or more computer readable media of claim 13 , wherein the candidate areas are determined by:
analyzing the image using a selected subject analysis technique to recognize the plurality of subjects of the image;
determining respective region scopes of the plurality of subjects based on coordinate information of boundary pixels of the plurality of subjects; and
selecting the respective r

Assignees

• Alibaba Group Holding Ltd

Inventors

• Pan Pan
• Liu Wei
• Li Min
• Zhang Yingya
• Hua Xiansheng

Classifications

• G06V20/70 Primary

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/806

  of extracted features · CPC title

• G06V10/809

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06F18/24323

  Tree-organised classifiers · CPC title

• G06F18/254

  of classification results, e.g. of results related to same input data · CPC title