Readability awareness in natural language processing systems

Patent metadata

Field	Value
Publication number	US-2017193092-A1
Application number	US-201615159986-A
Country	US
Kind code	A1
Filing date	May 20, 2016
Priority date	Jan 5, 2016
Publication date	Jul 6, 2017
Grant date	—

Abstract

Electronic natural language processing in a natural language processing (NLP) system, such as a Question-Answering (QA) system. A receives electronic text input, in question form, and determines a readability level indicator in the question. The readability level indicator includes at least a grammatical error, a slang term, and a misspelling type. The computer determines a readability level for the electronic text input based on the readability level indicator, and retrieves candidate answers based on the readability level.

First claim

What is claimed is:

1 . A method for electronic natural language processing in an electronic natural language processing (NLP) system, comprising:
receiving an electronic text input; determining a readability level indicator of the electronic text input, wherein the readability level indicator comprises at least one of a grammatical error, a slang term, and a misspelling type in the electronic text input; and determining a readability level of the electronic text input based on the readability level indicator.

2 . The method of claim 1 , further comprising:
receiving the electronic text input from an electronic input source in response to an input from a user.

3 . The method of claim 1 , further comprising:
identifying the electronic text input as a question.

4 . The method of claim 3 , further comprising:
generating a plurality of candidate answers for the question; and selecting a set of candidate answers from among the plurality of candidate answers based on matching readability levels of the set of candidate answers to the readability level of the question.

5 . The method of claim 1 , further comprising:
parsing the electronic text input using a full parsing process.

6 . The method of claim 1 , further comprising:
comparing the readability indicators of the electronic text input with readability indicators of one or more questions in a corpus of questions, wherein determining the readability level for the electronic text input is based on readability levels of the one or more questions.

7 . The method of claim 6 , further comprising:
training a natural language processing model based on the results of the comparison.

8 . The method of claim 1 , wherein identifying at least one of a slang term and a misspelling in the electronic text input comprises identifying, in the electronic text input, one or more of:
an abbreviation of a word corresponding to an acronym associated with the word in a collection of text messaging acronyms; and a misspelling of the word, wherein the misspelling corresponds to a phonetic reading of the word.

Assignees

• IBM

Inventors

• Byron Donna K
• Goyal Devendra
• Krishnamurthy Lakshminarayanan
• Santos Moraes Priscilla
• Smith Michael C

Classifications

• G06F16/3344 Primary

  using natural language analysis · CPC title

• G06F40/279

  Recognition of textual entities · CPC title

• G06F40/205

  Parsing · CPC title

• G06F40/253

  Grammatical analysis; Style critique · CPC title

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title