Analogy engine for grounding and situational awareness

Patent metadata

Field	Value
Publication number	US-11373069-B2
Application number	US-202016824917-A
Country	US
Kind code	B2
Filing date	Mar 20, 2020
Priority date	Mar 20, 2020
Publication date	Jun 28, 2022
Grant date	Jun 28, 2022

Abstract

An analogy generating system includes one or more image databases that include a first set of images depicting a first symbolic class and a second set of images depicting a second symbolic class and an autoencoder that receive images from the first set of images and the second set of images; determines a first characteristic shared between the first symbolic class and the second symbolic class using a first node from multiple nodes on a neural network; determine a second characteristic shared between the first symbolic class and the second symbolic class using a second node from multiple nodes on the neural network; and exchange the first characteristic and the second characteristic between the first node and the second node to establish an analogy between the first symbolic class and the second symbolic class.

First claim

What is claimed is:

1. An analogy generating system, comprising:
one or more image databases comprising a first set of images depicting a first symbolic class and a second set of images depicting a second symbolic class; and
an autoencoder configured to:
receive images from the first set of images and the second set of images;
determine a first characteristic of the first symbolic class using a neural network;
determine the first characteristic of the second symbolic class using the neural network;
determine an analogy between the first symbolic class and the second symbolic class using the first characteristic;
determine a first concept about the first symbolic class using the analogy;
determine a second concept about the second symbolic class using the analogy;
establish a second analogy between the first concept and a third symbolic class;
establish a third analogy between the second concept and the third symbolic class; and
determine a third concept about the third symbolic class using the second analogy and the third analogy.

2. The analogy generating system of claim 1 , wherein the analogy is a mapping between the first characteristic that defines how the first symbolic class and the second symbolic class are similar.

3. The analogy generating system of claim 1 , wherein the autoencoder comprises:
an input layer that receives the images;
an output layer;
a middle layer comprising a plurality of nodes, wherein the middle layer produces the analogy, and wherein the middle layer is compressed to be smaller than the input layer and the output layer.

4. The analogy generating system of claim 1 , wherein an incomplete image missing a shared characteristic between the first symbolic class and the second symbolic class is one of the images received by the autoencoder.

5. The analogy generating system of claim 4 , wherein the autoencoder adds the shared characteristic to the incomplete image based on the analogy between the first symbolic class and the second symbolic class.

6. The analogy generating system of claim 1 , wherein the first concept is a salient feature of the first symbolic class.

7. The analogy generating system of claim 1 , wherein the first set of images of the first symbolic class depicts a first action, a first face or facial feature, a first object, or a combination thereof.

8. The analogy generating system of claim 1 , wherein the second set of images of the second symbolic class depicts a second action, a second face or facial feature, a second object, or a combination thereof.

9. A method, comprising:
receiving, via a processor, a first set of images from a first image data base and a second set of images from a second image data base, wherein the first set of images depicts a first symbolic class and the second set of images depicts a second symbolic class;
determining, via the processor, a first characteristic shared between the first symbolic class and the second symbolic class using a first node and a second node from a plurality of nodes on a neural network;
exchanging, via the processor, the first characteristic between the first node and the second node to establish an analogy between the first symbolic class and the second symbolic class;
determining, via the processor, a first concept about the first symbolic class and a second concept about the second symbolic class using the analogy;
establishing, via the processor, a second analogy between the first concept and a third symbolic class;
establishing, via the processor, a third analogy between the second concept and the third symbolic class; and
determining, via the processor, a third concept about the third symbolic class using the second analogy and the third analogy.

10. The method of claim 9 , wherein the analogy is a mapping between the first characteristic that defines how the first symbolic class and the second symbolic class are similar.

11. The method of claim 9 , comprising receiving an incomplete image missing a shared characteristic between the first symbolic class and the second symbolic class, wherein the incomplete image is one of an image from the first and second set of images.

12. The method of claim 11 , comprising adding the shared characteristic to the incomplete image based on the analogy between the first symbolic class and the second symbolic class.

13. The method of claim 11 , wherein the first concept is a salient feature of the first symbolic class.

14. A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to:
receive a first set of images and a second set of images, wherein the first set of images depicts a first symbolic class and the second set of images depicts a second symbolic class;
determine a first characteristic shared between the first symbolic class and the second symbolic class using a neural network;
establish an analogy between the first symbolic class and the second symbolic class using the first characteristic;
determine a first concept about the first symbolic class and a second concept about the second symbolic class using the analogy;
establish a second analogy between the first concept and a third symbolic class and establish a third analogy between the second concept and the third symbolic class; and
determine a third concept about the third symbolic class using the second analogy and the third analogy.

15. The non-transitory computer readable medium of claim 14 , wherein the analogy is a mapping between the first characteristic that defines how the first symbolic class and the second symbolic class are similar.

16. The non-transitory computer readable medium of claim 14 , wherein the instructions, when executed by the processor, cause the processor to:
receive an incomplete image missing a shared characteristic between the first symbolic class and the second symbolic class, wherein the incomplete image is one of an image from the first and second set of images; and
add the shared characteristic to the incomplete image based on the analogy between the first symbolic class and the second symbolic class.

Assignees

• Gen Electric

Inventors

• Tu Peter Henry
• Gao Tao
• Chen Alexander S-Ban
• Tu Jilin

Classifications

• G06V40/16

  Human faces, e.g. facial parts, sketches or expressions · CPC title

• G06V10/764 Primary

  using classification, e.g. of video objects · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06F18/256

  of results relating to different input data, e.g. multimodal recognition · CPC title

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title