System and method for coupled detection of syntax and semantics for natural language understanding and generation

Patent metadata

Field	Value
Publication number	US-2019384824-A1
Application number	US-201916557520-A
Country	US
Kind code	A1
Filing date	Aug 30, 2019
Priority date	Nov 17, 2016
Publication date	Dec 19, 2019
Grant date	—

Abstract

A method includes performing, with at least one processing device, natural language understanding by iteratively (i) generating a semantic word and clause representation and (ii) generating a syntax. The generation of the semantic word and clause representation and the generation of the syntax occur iteratively such that (i) semantics are calculated from syntax by aggregating weights of syntactically-labeled context in which words or clauses appear and (ii) syntax is calculated from semantics by grouping common pairs of words or clauses with similar semantic relations, thereby producing a self-consistent coupled notion of syntax and semantics.

First claim

What is claimed is:

1 . A method comprising:
performing, with at least one processing device, natural language understanding by iteratively (i) generating a semantic word and clause representation and (ii) generating a syntax; wherein the generation of the semantic word and clause representation and the generation of the syntax occur iteratively such that (i) semantics are calculated from syntax by aggregating weights of syntactically-labeled context in which words or clauses appear and (ii) syntax is calculated from semantics by grouping common pairs of words or clauses with similar semantic relations, thereby producing a self-consistent coupled notion of syntax and semantics.

2 . The method of claim 1 , wherein generating the semantic word and clause representation comprises:
obtaining an initial syntax represented by a parse tree; weighting each item of syntactically-labeled context based on a number of occurrences; and weighting each occurrence of each item of syntactically-labeled context based on a decaying function of distance along the parse tree.

3 . The method of claim 2 , wherein generating the semantic word and clause representation further comprises:
using amplifiers along the parse tree, each amplifier representing a specific word or clause that strengthens, weakens, or inverts a relationship between two other words or clauses, each amplifier having a positive or negative multiplier for a weight of the associated relationship.

4 . The method of claim 3 , wherein generating the semantic word and clause representation further comprises:
measuring sentiments of words or clauses based on emotional context items and the amplifiers.

5 . The method of claim 1 , further comprising:
determining parts of speech for words and clauses by calculating inner products of semantic vectors and semantic co-vectors, the semantic vectors and semantic co-vectors comprising coefficients corresponding to items of syntactically-labeled context, the semantic co-vectors comprising coefficients that are repeated across all items of syntactically-labeled context having a common part of speech.

6 . The method of claim 1 , wherein generating the syntax comprises:
producing a parse tree and a frequency table of semantic relations that represent syntactic constructs.

7 . The method of claim 6 , wherein generating the syntax further comprises:
using an auto-associative technique to identify a memory cell for storing a semantic relation by finding an angularly nearest vector for a semantic vector.

8 . An apparatus comprising:
at least one processing device; and at least one memory storing instructions that, when executed by the at least one processing device, cause the at least one processing device to perform natural language understanding by iteratively (i) generating a semantic word and clause representation and (ii) generating a syntax; wherein the generation of the semantic word and clause representation and the generation of the syntax occur iteratively such that the at least one processing device is configured to (i) calculate semantics from syntax by aggregating weights of syntactically-labeled context in which words or clauses appear and (ii) calculate syntax from semantics by grouping common pairs of words or clauses with similar semantic relations, thereby producing a self-consistent coupled notion of syntax and semantics.

9 . The apparatus of claim 8 , wherein the instructions that when executed cause the at least one processing device to generate the semantic word and clause representation comprise:
instructions that when executed cause the at least one processing device to:
obtain an initial syntax represented by a parse tree;
weight each item of syntactically-labeled context based on a number of occurrences; and
weight each occurrence of each item of syntactically-labeled context based on a decaying function of distance along the parse tree.

10 . The apparatus of claim 9 , wherein the instructions that when executed cause the at least one processing device to generate the semantic word and clause representation further comprise:
instructions that when executed cause the at least one processing device to use amplifiers along the parse tree, each amplifier representing a specific word or clause that strengthens, weakens, or inverts a relationship between two other words or clauses, each amplifier having a positive or negative multiplier for a weight of the associated relationship.

11 . The apparatus of claim 10 , wherein the instructions that when executed cause the at least one processing device to generate the semantic word and clause representation further comprise:
instructions that when executed cause the at least one processing device to measure sentiments of words or clauses based on emotional context items and the amplifiers.

12 . The apparatus of claim 8 , wherein the instructions when executed further cause the at least one processing device to:
determine parts of speech for words and clauses based on inner products of semantic vectors and semantic co-vectors, the semantic vectors and semantic co-vectors comprising coefficients corresponding to items of syntactically-labeled context, the semantic co-vectors comprising coefficients that are repeated across all items of syntactically-labeled context having a common part of speech.

13 . The apparatus of claim 8 , wherein the instructions that when executed cause the at least one processing device to generate the syntax comprise:
instructions that when executed cause the at least one processing device to produce a parse tree and a frequency table of semantic relations that represent syntactic constructs.

14 . The apparatus of claim 13 , wherein the instructions that when executed cause the at least one processing device to generate the syntax further comprise:
instructions that when executed cause the at least one processing device to use an auto-associative technique to identify a memory cell for storing a semantic relation by finding an angularly nearest vector for a semantic vector.

15 . A non-transitory computer readable medium containing instructions that, when executed by at least one processing device, cause the at least one processing device to:
perform natural language understanding by iteratively (i) generating a semantic word and clause representation and (ii) generating a syntax; wherein the generation of the semantic word and clause representation and the generation of the syntax occur iteratively such that the at least one processing device is configured to (i) calculate semantics from syntax by aggregating weights of syntactically-labeled context in which words or clauses appear and (ii) calculate syntax from semantics by grouping common pairs of words or clauses with similar semantic relations, thereby producing a self-consistent coupled notion of syntax and semantics.

16 . The non-transitory computer readable medium of claim 15 , wherein the instructions that when executed cause the at least one processing device to generate the semantic word and clause representation comprise:
instructions that when executed cause the at least one processing device to:
obtain an initial syntax represented by a parse tree;
weight each item of syntactically-labeled context based on a number of occurrences; and
weight each occurrence of each item of syntactically-labeled context based on a decaying function of distance along the parse tree.

17 . The non-transitory computer readable medium of claim 16 , wherein the instructions that when executed cause the

Assignees

• Goldman Sachs & Co Llc

Inventors

• Burchard Paul

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G06F40/284 Primary

  Lexical analysis, e.g. tokenisation or collocates · CPC title

• G06F40/56 Primary

  Natural language generation · CPC title

• G06F40/211

  Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars · CPC title

• G06F17/2881 Primary

  Physics · mapped topic