Normalizing electronic communications using a neural-network normalizer and a neural-network flagger

Patent metadata

Field	Value
Publication number	US-9552547-B2
Application number	US-201514937810-A
Country	US
Kind code	B2
Filing date	Nov 10, 2015
Priority date	May 29, 2015
Publication date	Jan 24, 2017
Grant date	Jan 24, 2017

Abstract

Electronic communications can be normalized using neural networks. For example, an electronic representation of a noncanonical communication can be received. A normalized version of the noncanonical communication can be determined using a normalizer including a neural network. The neural network can receive a single vector at an input layer of the neural network and transform an output of a hidden layer of the neural network into multiple values that sum to a total value of one. Each value of the multiple values can be a number between zero and one and represent a probability of a particular character being in a particular position in the normalized version of the noncanonical communication. The neural network can determine the normalized version of the noncanonical communication based on the multiple values. Whether the normalized version should be output can be determined based on a result from a flagger including another neural network.

First claim

What is claimed is:

1. A non-transitory computer readable medium comprising program code executable by a processor for causing the processor to:
receive an electronic representation of a noncanonical communication;
feed a vector that is representative of the noncanonical communication as input to a normalizer, wherein the normalizer is operable for implementing a first neural network to:
receive the vector at an input layer of the first neural network,
perform matrix operations on the vector using a plurality of hidden layers to generate hidden values,
provide hidden values from a final hidden layer of the plurality of hidden layers to an output layer of the first neural network,
perform, at the output layer, a softmax operation on the hidden values to generate a plurality of values representing probabilities of particular characters being in particular positions in a normalized version of the noncanonical communication, and
determine the normalized version of the noncanonical communication based on the plurality of values, wherein the normalized version of the noncanonical communication is different from the noncanonical communication;

feed the vector as input to a flagger for operating a second neural network that is trained separately from the first neural network;
determine, based on a result from the flagger, that the normalized version of the noncanonical communication should be outputted; and
output the normalized version of the noncanonical communication or a corrected version of the normalized version of the noncanonical communication, wherein the corrected version of the noncanonical communication is different from the noncanonical communication.

2. The non-transitory computer readable medium of claim 1 , further comprising program code executable by the processor for causing the processor to:
in response to determining that the normalized version of the noncanonical communication should be outputted based on the result from the flagger:
output the normalized version of the noncanonical communication in response to determining that the normalized version of the noncanonical communication is present in a database; and
output the corrected version of the normalized version of the noncanonical communication in response to determining that the normalized version of the noncanonical communication is not present in the database.

3. The non-transitory computer readable medium of claim 2 , further comprising program code executable by the processor for causing the processor to:
determine the corrected version using a conformer, the conformer configured to:
determine a Levenshtein distance between the normalized version of the noncanonical communication and each word of a plurality of words in the database; and
select as the corrected version a word from the plurality of words in the database associated with a smallest Levenshtein distance.

4. The non-transitory computer readable medium of claim 1 , further comprising program code executable by the processor for causing the processor to:
preprocess the noncanonical communication prior to determining the normalized version of the noncanonical communication by:
determining a plurality of vectors associated with the noncanonical communication by transforming each character in the noncanonical communication into a vector comprising a predetermined length; and
concatenating the plurality of vectors together into the vector, the vector comprising another predetermined length.

5. The non-transitory computer readable medium of claim 1 , wherein each hidden layer of the plurality of hidden layers comprises a layer of units between the input layer and the output layer of the first neural network.

6. The non-transitory computer readable medium of claim 1 , wherein the first neural network is configured so that every unit of the first neural network only propagates an output value to a subsequent layer of the first neural network.

7. The non-transitory computer readable medium of claim 1 , wherein the input layer of the first neural network is a first input layer, the output layer of the first neural network is a first output layer, the softmax operation is a first softmax operation, and wherein the second neural network of the flagger is configured to:
receive the vector at a second input layer of the second neural network;
perform a plurality of matrix operations on the vector using at least two hidden layers of the second neural network to generate a plurality of hidden values, wherein each hidden layer of the at least two hidden layers comprises a layer of units between the second input layer and a second output layer of the second neural network;
provide one or more hidden values of the plurality of hidden values to the second output layer of the second neural network;
perform, at the second output layer, a second softmax operation on the one or more hidden values to generate a first value indicating a first probability that the normalized version of the noncanonical communication should be output and a second value indicating a second probability that the normalized version of the noncanonical communication should not be output; and
determine that the normalized version of the noncanonical communication should be output in response to the first value being greater than the second value, and that the normalized version of the noncanonical communication should not be output in response to the second value being greater than the first value.

8. The non-transitory computer readable medium of claim 1 , further comprising program code executable by the processor for causing the processor to:
determine the normalized version of the noncanonical communication using the normalizer simultaneously and in parallel to determining that the normalized version of the noncanonical communication should be outputted using the flagger.

9. The non-transitory computer readable medium of claim 1 , further comprising program code executable by the processor for causing the processor to:
include the normalized version of the noncanonical communication in a data set for use in textual analysis; and
perform textual analysis on the data set to determine one or more trends indicated by the data set.

10. A method comprising:
receiving an electronic representation of a noncanonical communication;
feeding a vector that is representative of the noncanonical communication as input to a normalizer, wherein the normalizer implements a first neural network to:
receive the vector at an input layer of the first neural network,
perform matrix operations on the vector using a plurality of hidden layers to generate hidden values,
provide hidden values from a final hidden layer of the plurality of hidden layers to an output layer of the first neural network,
perform, at the output layer, a softmax operation on the hidden values to generate a plurality of values representing probabilities of particular characters being in particular positions in a normalized version of the noncanonical communication, and
determine the normalized version of the noncanonical communication based on the plurality of values, wherein the normalized version of the noncanonical communication is different from the noncanonical communication;

feeding the vector as input to a flagger that operates a second neural network that is trained separately from the first neural network;
determining, based on a result from the flagger, that the normalized version of the noncanonical communication should be outputted; and
outputting the normalized version of the noncanonical communication or a corrected version of the normalized version of the noncanonical communication, wherein the corrected version of

Assignees

• Sas Inst Inc
• Univ North Carolina State

Inventors

• Leeman-Munk Samuel Paul
• Min Wookhee
• Mott Bradford Wayne
• Lester Ii James Curtis
• Cox James Allen

Classifications

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06F40/232

  Orthographic correction, e.g. spell checking or vowelisation · CPC title

• G06F40/30

  Semantic analysis · CPC title

• H04L51/063

  Content adaptation, e.g. replacement of unsuitable content · CPC title