Cognitive integrated image classification and annotation

Patent metadata

Field	Value
Publication number	US-2018342092-A1
Application number	US-201715606019-A
Country	US
Kind code	A1
Filing date	May 26, 2017
Priority date	May 26, 2017
Publication date	Nov 29, 2018
Grant date	—

Abstract

A system for classifying and tagging digital images includes: a CPU, a computer readable memory, and a computer readable storage medium associated with a computer device; program instructions defining plural pipelines each configured to classify and tag aspects of the image, wherein a first one of the plural pipelines is configured to classify and tag an object in the image, and a second one of the plural pipelines is configured to classify and tag a kinematic aspect of the object in the image; program instructions defining a controller configured to: pass the image to each of the plural pipelines in a predefined order; and output an annotation of the image to a user interface. The program instructions are stored on the computer readable storage medium for execution by the CPU via the computer readable memory.

First claim

1 .- 10 . (canceled)

11 . A system for classifying and tagging digital images, comprising:
a CPU, a computer readable memory, and a computer readable storage medium associated with a computer device; program instructions defining plural pipelines each configured to classify and tag aspects of the image, wherein a first one of the plural pipelines is configured to classify and tag an object in the image, and a second one of the plural pipelines is configured to classify and tag a kinematic aspect of the object in the image; program instructions defining a controller configured to: pass the image to each of the plural pipelines in a predefined order; and output an annotation of the image to a user interface, wherein the program instructions are stored on the computer readable storage medium for execution by the CPU via the computer readable memory.

12 . The system of claim 11 , wherein the plural pipelines comprise:
a pipeline configured to classify and tag an aggregation of objects in the image; a pipeline configured to classify and tag an aggregation of the kinematic aspects of the objects in the image; and a pipeline configured to classify and tag a situation in the image.

13 . The system of claim 11 , wherein the first one of the plural pipelines classifies the object by comparing a shape of the object to predefined object templates.

14 . The system of claim 11 , wherein the second one of the plural pipelines classifies the kinematic aspect of the object by comparing a shape of the object to predefined kinematic templates.

15 . The system of claim 11 , wherein the controller is configured to:
obtain an insight about the object in the image from a big data platform; and adjust an object tag of the image based on the insight.

16 . The system of claim 15 , wherein the adjusting the object tag comprises replacing the object tag with one of a name, a relationship, and an age descriptor.

17 . The system of claim 11 , wherein the image comprises a sequence of plural images, and the controller is configured to:
pass each one of the plural images to each of the plural pipelines; and eliminate redundant tags from consecutive ones of the plural images.

18 . A computer program product for classifying and tagging digital images, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer device to cause the computer device to:
receive an input of an image from a user interface; tag the image with at least one object tag using an object tagging pipeline; tag the image with at least one kinematic tag using a kinematic tagging pipeline; obtain at least one insight about the image from a big data platform; tag the image with at least one personalized object tag based on at least one object tag and the at least one insight; generate an annotation of the image based on the at least one personalized object tag and the at least one kinematic tag; and output the annotation to the user interface.

19 . The computer program product of claim 18 , wherein the object tagging pipeline classifies the at least one object by comparing a shape of the at least one object to predefined object templates.

20 . The computer program product of claim 18 , wherein the kinematic tagging pipeline classifies a kinematic aspect of the at least one object by comparing a shape of the at least one object to predefined kinematic templates.

Assignees

• IBM

Inventors

• Choo Kristina Y
• Hodge Rashida A
• Ramachandran Krishnan K
• Sivakumar Gandhi

Classifications

• G06V20/70 Primary

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F16/5854

  using shape and object relationship · CPC title