Knowledge-graph biased classification for data

Patent metadata

Field	Value
Publication number	US-10474949-B2
Application number	US-201414528890-A
Country	US
Kind code	B2
Filing date	Oct 30, 2014
Priority date	Aug 19, 2014
Publication date	Nov 12, 2019
Grant date	Nov 12, 2019

Abstract

A method for classifying an object includes applying multiple confidence values to multiple objects. The method also includes determining a metric based on the multiple confidence values. The method further includes determining a classification of a first object from the multiple objects based on a knowledge-graph when the metric is above a threshold.

First claim

What is claimed is:

1. A method for computer-implemented tagging of objects in an image, the method comprising:
processing the image by a computer-implemented artificial neural network to identify at least a first and a second object and generate, for at least the first object, at least a first and a second predicted classification and corresponding confidence values, the first object being partially visible in the image;
generating an adjusted first confidence value by adjusting the confidence value corresponding to the first predicted classification based on a predetermined co-existence probability for the first predicted classification and the second object and also at least one of a physical location of an image capturing device or a time when the image was captured;
generating an adjusted second confidence value by adjusting the confidence value corresponding to the second predicted classification based on a predetermined co-existence probability for the second predicted classification and the second object and also at least one of the physical location of the image capturing device or the time when the image was captured;
determining a classification for the first object based on the first and second predicted classifications and the corresponding adjusted first and second confidence values; and
generating a tag for the first object using the determined classification.

2. The method of claim 1 , wherein the predetermined co-existence probabilities are obtained from a knowledge-graph matrix.

3. The method of claim 2 , further comprising updating the knowledge-graph matrix based on the classification of the first object.

4. The method of claim 3 , wherein the updating is based on user input.

5. The method of claim 1 , further comprising calculating a classifier confusion value based on a difference between the confidence values corresponding to the first and second predicted classifications.

6. The method of claim 5 , wherein the steps of generating adjusted confidence values are performed only if the classifier confusion value is greater than a threshold.

7. The method of claim 1 , wherein the image is a frame of a video stream.

8. A computer apparatus comprising a memory and a processor coupled to the memory, wherein the processor is adapted to:
process an image by a computer-implemented artificial neural network to identify at least a first and a second object and generate, for at least the first object, at least a first and a second predicted classification and corresponding confidence values, the first object being partially visible in the image;
generate an adjusted first confidence value by adjusting the confidence value corresponding to the first predicted classification based on a predetermined co-existence probability for the first predicted classification and the second object and also at least one of a physical location of an image capturing device or a time when the image was captured;
generate an adjusted second confidence value by adjusting the confidence value corresponding to the second predicted classification based on a predetermined co-existence probability for the second predicted classification and the second object and also at least one of the physical location of the image capturing device or the time when the image was captured;
determine a classification for the first object based on the first and second predicted classifications and the corresponding adjusted first and second confidence values; and
generate a tag for the first object using the determined classification.

9. The apparatus of claim 8 , wherein the predetermined co-existence probabilities are obtained from a knowledge-graph matrix.

10. The apparatus of claim 9 , wherein the processor is further adapted to update the knowledge-graph matrix based on the classification of the first object.

11. The apparatus of claim 10 , wherein the updating is based on user input.

12. The apparatus of claim 8 , wherein the processor is further adapted to calculate a classifier confusion value based on a difference between the confidence values corresponding to the first and second predicted classifications.

13. The apparatus of claim 12 , wherein the processor is adapted to generate the adjusted confidence values only if the classifier confusion value is greater than a threshold.

14. The apparatus of claim 8 , wherein the image is a frame of a video stream.

15. A non-transitory computer-readable medium having program code recorded thereon for computer-implemented tagging of objects in an image, the program code being executed by a processor and comprising:
program code to process the image by a computer-implemented artificial neural network to identify at least a first and a second object and generate, for at least the first object, at least a first and a second predicted classification and corresponding confidence values, the first object being partially visible in the image;
program code to generate an adjusted first confidence value by adjusting the confidence value corresponding to the first predicted classification based on a predetermined co-existence probability for the first predicted classification and the second object and also at least one of a physical location of an image capturing device or a time when the image was captured;
program code to generate an adjusted second confidence value by adjusting the confidence value corresponding to the second predicted classification based on a predetermined co-existence probability for the second predicted classification and the second object and also at least one of the physical location of the image capturing device or the time when the image was captured;
program code to determine a classification for the first object based on the first and second predicted classifications and the corresponding adjusted first and second confidence values; and
program code to generate a tag for the first object using the determined classification.

Assignees

• Qualcomm Inc

Inventors

• Majumdar Somdeb
• Towal Regan Blythe
• Talathi Sachin Subhash
• Julian David Jonathan
• Annapureddy Venkata Sreekanta Reddy

Classifications

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title

• G06N3/042 Primary

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/08 Primary

  Learning methods · CPC title