Analogy engine for grounding and situational awareness

Patent metadata

Field	Value
Publication number	US-2021295102-A1
Application number	US-202016824917-A
Country	US
Kind code	A1
Filing date	Mar 20, 2020
Priority date	Mar 20, 2020
Publication date	Sep 23, 2021
Grant date	—

Abstract

An analogy generating system includes one or more image databases that include a first set of images depicting a first symbolic class and a second set of images depicting a second symbolic class and an autoencoder that receive images from the first set of images and the second set of images; determines a first characteristic shared between the first symbolic class and the second symbolic class using a first node from multiple nodes on a neural network; determine a second characteristic shared between the first symbolic class and the second symbolic class using a second node from multiple nodes on the neural network; and exchange the first characteristic and the second characteristic between the first node and the second node to establish an analogy between the first symbolic class and the second symbolic class.

First claim

What is claimed is:

1 . An analogy generating system comprising:
one or more image databases comprising a first set of images depicting a first symbolic class and a second set of images depicting a second symbolic class; and an autoencoder configured to:
receive images from the first set of images and the second set of images;
determine a first characteristic of the first symbolic class using a neural network;
determine the first characteristic of the second symbolic class using the neural network; and
determine an analogy between the first symbolic class and the second symbolic class using the first characteristic.

2 . The analogy generating system of claim 1 , wherein the analogy is a mapping between the first characteristic that defines how the first symbolic class and the second symbolic class are similar.

3 . The analogy generating system of claim 1 , wherein the autoencoder comprises:
an input layer that receives the images; an output layer; a middle layer comprising a plurality of nodes, wherein the middle layer produces the analogy, and wherein the middle layer is compressed to be smaller than the input layer and the output layer.

4 . The analogy generating system of claim 1 , wherein an incomplete image missing a shared characteristic between the first symbolic class and the second symbolic class is one of the images received by the autoencoder.

5 . The analogy generating system of claim 4 , wherein the autoencoder adds the shared characteristic to the incomplete image based on the analogy between the first symbolic class and the second symbolic class.

6 . The analogy generating system of claim 1 , wherein the autoencoder is configured to:
determine a first concept about the first symbolic class using the analogy; determine a second concept about the second symbolic class using the analogy; establish a second analogy between the first concept and a third symbolic class; establish a third analogy between the second concept and the third symbolic class; and determine a third concept about the third symbolic class using the second analogy and the third analogy.

7 . The analogy generating system of claim 6 , wherein the first concept is a salient feature of the first symbolic class.

8 . The analogy generating system of claim 1 , wherein the first set of images of the first symbolic class depicts a first action, a first face or facial feature, a first object, or a combination thereof.

9 . The analogy generating system of claim 1 , wherein the second set of images of the second symbolic class depicts a second action, a second face or facial feature, a second object, or a combination thereof.

10 . A method comprising:
receiving, via a processor, a first set of images from a first image data base and a second set of images from a second image data base, wherein the first set of images depicts a first symbolic class and the second set of images depicts a second symbolic class; determining, via the processor, a first characteristic shared between the first symbolic class and the second symbolic class using a first node and a second node from a plurality of nodes on a neural network; and exchanging, via the processor, the first characteristic between the first node and the second node to establish an analogy between the first symbolic class and the second symbolic class.

11 . The method of claim 10 , wherein the analogy is a mapping between the first characteristic that defines how the first symbolic class and the second symbolic class are similar.

12 . The method of claim 10 , wherein an incomplete image missing a shared characteristic between the first symbolic class and the second symbolic class is one of an image from the first and second set of images received by the autoencoder.

13 . The method of claim 12 , wherein the autoencoder adds the shared characteristic to the incomplete image based on the analogy between the first symbolic class and the second symbolic class.

14 . The method of claim 10 , comprising:
determining a first concept about the first symbolic class using the analogy; determining a second concept about the second symbolic class using the analogy; establishing a second analogy between the first concept and a third symbolic class; establishing a third analogy between the second concept and the third symbolic class; and determining a third concept about the third symbolic class using the second analogy and the third analogy.

15 . The method of claim 14 , wherein the first concept is a salient feature of the first symbolic class.

16 . A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to:
receive a first set of images and a second set of images, wherein the first set of images depicts a first symbolic class and the second set of images depicts a second symbolic class; determine a first characteristic shared between the first symbolic class and the second symbolic class using a neural network; and establish an analogy between the first symbolic class and the second symbolic class using the first characteristic.

17 . The non-transitory computer readable medium of claim 16 , wherein the analogy is a mapping between the first characteristic that defines how the first symbolic class and the second symbolic class are similar.

18 . The non-transitory computer readable medium of claim 16 , wherein an incomplete image missing a shared characteristic between the first symbolic class and the second symbolic class is one of an image from the first and second set of images received by the autoencoder.

19 . The non-transitory computer readable medium of claim 16 , wherein the autoencoder adds the shared characteristic to the incomplete image based on the analogy between the first symbolic class and the second symbolic class.

20 . The non-transitory computer readable medium of claim 16 , wherein the processor:
determines a first concept about the first symbolic class using the analogy; determines a second concept about the second symbolic class using the analogy; establishes a second analogy between the first concept and a third symbolic class; establishes a third analogy between the second concept and the third symbolic class; and determines a third concept about the third symbolic class using the second analogy and the third analogy.

Assignees

• Gen Electric

Inventors

• Tu Peter Henry
• Gao Tao
• Chen Alexander S-Ban
• Tu Jilin

Classifications

• G06V40/16

  Human faces, e.g. facial parts, sketches or expressions · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06F18/256

  of results relating to different input data, e.g. multimodal recognition · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title