System and method for sentiment lexicon expansion

Patent metadata

Field	Value
Publication number	US-10089296-B2
Application number	US-201514984498-A
Country	US
Kind code	B2
Filing date	Dec 30, 2015
Priority date	Dec 30, 2015
Publication date	Oct 2, 2018
Grant date	Oct 2, 2018

Abstract

Systems and methods for sentiment lexicon expansion receive at least a domain specific corpus comprising a plurality of words, and a generic sentiment lexicon; parse the plurality of words in the domain specific corpus into a plurality of dependency relations; identify, using one or more syntactic dependency rules and at least one of the plurality of dependency relations, a set of one or more sentiment candidates in the domain specific corpus; filter from the set of one or more sentiment candidates any sentiment candidate having an expected performance below a predefined threshold; sample the filtered set of one or more sentiment candidates to be used in a qualitative evaluation; and, for each sentiment candidate that passes the qualitative evaluation, add the sentiment candidate to the generic sentiment lexicon.

First claim

The invention claimed is:

1. A method for sentiment lexicon expansion, performed on a computing device having a processor, memory, and one or more code sets stored in the memory and executing in the processor, the method comprising:
receiving, by the processor, at least a domain specific corpus comprising a plurality of words, and a generic sentiment lexicon;
parsing, by the processor, the plurality of words in the domain specific corpus into a plurality of dependency relations;
identifying, by the processor, using one or more syntactic dependency rules and at least one of the plurality of dependency relations, a set of one or more sentiment candidates in the domain specific corpus;
wherein the one or more syntactic dependency rules comprises at least one rule in which:
if a pair of adverbs found in the domain specific corpus have either an ‘and’ conjunction dependency relationship or a ‘but’ conjunction dependency relationship, wherein a first adverb of the pair of adverbs was previously included in the generic sentiment lexicon and a second adverb of the pair of adverbs was not previously included in the generic sentiment lexicon, then the second adverb is identified as a sentiment candidate;

filtering, by the processor, from the set of one or more sentiment candidates any sentiment candidate having an expected performance below a predefined threshold;
sampling, by the processor, the filtered set of one or more sentiment candidates to be used in a qualitative evaluation; and
for each sentiment candidate that passes the qualitative evaluation, adding, by the processor, the sentiment candidate to the generic sentiment lexicon.

2. The method as in claim 1 , wherein the second adverb is further identified as having a same polarity as a polarity of the first adverb when the first adverb and the second adverb have an ‘and’ conjunction dependency relationship; and
wherein the second adverb is further identified as having an opposite polarity to the polarity of the first adverb when the first adverb and the second adverb have a ‘but’ conjunction dependency relationship.

3. The method as in claim 1 , wherein the one or more syntactic dependency rules comprises at least one rule in which:
an adverb found in the domain specific corpus which has not been previously included in the generic sentiment lexicon and which modifies a known feature found in the domain specific corpus is identified as a sentiment candidate.

4. The method as in claim 1 , wherein the one or more syntactic dependency rules comprises at least one rule in which:
an adverb, found in the domain specific corpus that has not been previously included in the generic sentiment lexicon, which modifies a context word found in the domain specific corpus, the context word having as its subject a known feature found in the domain specific corpus, is identified as a sentiment candidate.

5. The method as in claim 1 , further comprising:
identifying a polarity of one or more of the one or more sentiment candidates, wherein the polarity is one of positive and negative.

6. The method as in claim 5 , further comprising:
identifying an intensity of the polarity of the one or more of the one or more sentiment candidates.

7. The method as in claim 1 , further comprising:
for each of the one or more sentiment candidates:
if a polarity of the sentiment can be determined by applying the one or more syntactic dependency rules to the domain specific corpus, determining by the processor whether the polarity of the sentiment is positive or negative, and associating the sentiment with the determined polarity; and
if a polarity of the sentiment cannot be determined by applying the one or more syntactic dependency rules to the domain specific corpus, identifying the sentiment polarity as undetermined.

8. The method as in claim 1 , further comprising:
evaluating, by the processor, at least one of a recall value and a precision value for an expanded sentiment lexicon, wherein the expanded sentiment lexicon comprises the generic sentiment lexicon and each sentiment candidate added to the generic sentiment lexicon.

9. A system for sentiment lexicon expansion, comprising:
a processor;
a memory; and
one or more code sets stored in the memory and executing in the processor, which, when executed, configure the processor to:
receive at least a domain specific corpus comprising a plurality of words, and a generic sentiment lexicon;
parse the plurality of words in the domain specific corpus into a plurality of dependency relations;
identify, using one or more syntactic dependency rules and at least one of the plurality of dependency relations, a set of one or more sentiment candidates in the domain specific corpus;
wherein the one or more syntactic dependency rules comprises at least one rule in which:
if a pair of adverbs found in the domain specific corpus have either an ‘and’ conjunction dependency relationship or a ‘but’ conjunction dependency relationship, wherein a first adverb of the pair of adverbs was previously included in the generic sentiment lexicon and a second adverb of the pair of adverbs was not previously included in the generic sentiment lexicon, then the second adverb is identified as a sentiment candidate;

filter from the set of one or more sentiment candidates any sentiment candidate having an expected performance below a predefined threshold;
sample the filtered set of one or more sentiment candidates to be used in a qualitative evaluation; and
for each sentiment candidate that passes the qualitative evaluation, add the sentiment candidate to the generic sentiment lexicon.

10. The system of claim 9 , wherein the second adverb is further identified as having a same polarity as a polarity of the first adverb when the first adverb and the second adverb have an ‘and’ conjunction dependency relationship; and
wherein the second adverb is further identified as having an opposite polarity to the polarity of the first adverb when the first adverb and the second adverb have a ‘but’ conjunction dependency relationship.

11. The system of claim 9 , wherein the one or more syntactic dependency rules comprises at least one rule in which:
an adverb found in the domain specific corpus which has not been previously included in the generic sentiment lexicon and which modifies a known feature found in the domain specific corpus is identified as a sentiment candidate.

12. The system of claim 9 , wherein the one or more syntactic dependency rules comprises at least one rule in which:
an adverb, found in the domain specific corpus that has not been previously included in the generic sentiment lexicon, which modifies a context word found in the domain specific corpus, the context word having as its subject a known feature found in the domain specific corpus, is identified as a sentiment candidate.

13. The system of claim 9 , wherein the one or more code sets further configure the processor to:
identify a polarity of one or more of the one or more sentiment candidates, wherein the polarity is one of positive and negative.

14. The system of claim 13 , wherein the one or more code sets further configure the processor to:
Identify an intensity of the polarity of the one or more of the one or more sentiment candidates.

15. The system of claim 9 , wherein, for each of the one or more sentiment candidates, the one or more code sets further configure the processor to:
if a polarity of the sentiment can be determined by applying the one or more syntactic dependency rules to the domain specific corpus, determine whether the polarity of the sentiment is p

Assignees

• Nice Systems Ltd
• Nice Ltd

Inventors

• ZEICHNER Naomi
• Shafran Victor

Classifications

• G06F40/268

  Morphological analysis · CPC title

• G06F40/30

  Semantic analysis · CPC title

• G06F40/226

  Validation · CPC title

• G06F40/211 Primary

  Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars · CPC title

• G06F40/242

  Dictionaries · CPC title