Language entity identification

Patent metadata

Field	Value
Publication number	US-11170183-B2
Application number	US-201816132687-A
Country	US
Kind code	B2
Filing date	Sep 17, 2018
Priority date	Sep 17, 2018
Publication date	Nov 9, 2021
Grant date	Nov 9, 2021

Abstract

Methods, systems, and computer program products are provided for language entity identification. In one embodiment, a computer-implemented method is disclosed. In the method, respective pinyin codes may be determined for respective Chinese characters comprised in a string that is to be processed. Then, respective pinyin features may be generated from the respective pinyin codes. Next, a candidate language entity may be identified from the string based on the respective pinyin features and a mapping describing an association between pinyin features and language entity. In other embodiments, a computer-implemented system and a computer program product for security management are disclosed.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
determining, by one or more processors, respective pinyin codes for respective Chinese characters comprised in a string that is to be processed, the pinyin codes including an initial portion and a final portion;
generating, by the one or more processors, respective pinyin features from the respective pinyin codes, the pinyin features including the initial portion, the final portion, and tone marks associated with the Chinese characters; and
identifying, by the one or more processors, a candidate language entity representing a word from the string based on the respective pinyin features and a mapping, stored in computer memory, describing an association between pinyin features and language entities, the candidate language entity being a homophone of another language entity representing the word,
wherein the determination of respective pinyin codes further comprises:
with respect to a Chinese character comprised in the string, determining, by the one or more processors, the initial portion in a pinyin code for the Chinese character;
replacing, by the one or more processors, the initial portion with an updated initial portion that has a similar pronunciation as the initial portion; and
updating, by the one or more processors, the pinyin code based on the updated initial portion.

2. The computer-implemented method of claim 1 , wherein the determination of respective pinyin codes further comprises:
with respect to a Chinese character comprised in the string, determining, by the one or more processors, the tone mark associated with the Chinese character; and
updating, by the one or more processors, the pinyin code for the Chinese character based on the determined tone mark.

3. The computer-implemented method of claim 1 , wherein the determination of respective pinyin codes further comprises:
with respect to a Chinese character comprised in the string, determining, by the one or more processors, the final portion in a pinyin code for the Chinese character; and
replacing, by the one or more processors, the final portion with an updated final portion that has a similar pronunciation as the final portion;
updating, by the one or more processors, the pinyin code based on the updated final portion.

4. The computer-implemented method of claim 1 , wherein the generation of the respective pinyin features comprises:
with respect to a pinyin code, obtaining, by the one or more processors, a predefined length for generating a pinyin feature from the pinyin code;
generating, by the one or more processors, the pinyin feature based on at least one padding symbol and the pinyin code in response to a length of the pinyin code being below the predefined length.

5. The computer-implemented method of claim 1 , further comprising:
obtaining, by the one or more processors, a plurality of sample language entities;
with respect to one of the plurality of sample language entities, determining, by the one or more processors, respective sample pinyin codes for respective Chinese characters comprised in the sample language entity;
generating, by the one or more processors, respective sample pinyin features from the respective sample pinyin codes; and
training, by the one or more processors, the mapping based on the respective sample pinyin features and the sample language entity, such that the trained mapping identifies the sample language entity.

6. The computer-implemented method of claim 5 , wherein one of the sample language entities is labeled with a name type, and the training of the mapping further comprises training, by the one or more processors, the mapping based on the name type, such that the trained mapping identifies the sample language entity as the name type.

7. The computer-implemented method of claim 6 , further comprising providing, by the one or more processors, a candidate name type associated with the candidate language entity.

8. The computer-implemented method of claim 6 , wherein the obtaining of the plurality of sample language entities comprises selecting a sample language entity that is translated from another language, and wherein the name type comprises at least one of: a name of a person, a name of place, and a name of a drug.

9. A computer-implemented system comprising one or more processors coupled to a computer-readable memory unit, the memory unit comprising instructions that when executed by the computer processor implements a method comprising:
determining, by the one or more processors, respective pinyin codes for respective Chinese characters comprised in a string that is to be processed, the pinyin codes including an initial portion and a final portion;
generating, by the one or more processors, respective pinyin features from the respective pinyin codes, the pinyin features including the initial portion, the final portion, and tone marks associated with the Chinese characters; and
identifying, by the one or more processors, a candidate language entity representing a word from the string based on the respective pinyin features and a mapping, stored in computer memory, describing an association between pinyin features and language entities, the candidate language entity being a homophone of another language entity representing the word,
wherein the determination of respective pinyin codes further comprises:
with respect to a Chinese character comprised in the string, determining, by the one or more processors, the initial portion in a pinyin code for the Chinese character;
replacing, by the one or more processors, the initial portion with an updated initial portion that has a similar pronunciation as the initial portion; and
updating, by the one or more processors, the pinyin code based on the updated initial portion.

10. The computer-implemented system of claim 9 , wherein the determination of respective pinyin codes further comprises: with respect to a Chinese character comprised in the string,
determining, by the one or more processors, the tone mark associated with the Chinese character; and
updating, by the one or more processors, the pinyin code for the Chinese character based on the determined tone mark.

11. The computer-implemented system of claim 9 , wherein the determination of respective pinyin codes further comprises: with respect to a Chinese character comprised in the string,
determining, by the one or more processors, a final portion in a pinyin code for the Chinese character; and
updating, by the one or more processors, the pinyin code based on the determined final portion.

12. The computer-implemented system of claim 9 , wherein the generation of the respective pinyin features comprises:
with respect to a pinyin code, obtaining, by the one or more processors, a predefined length for generating a pinyin feature from the pinyin code;
generating, the by one or more processors, the pinyin feature based on at least one padding symbol and the pinyin code in response to a length of the pinyin code being below the predefined length.

13. The computer-implemented system of claim 9 , further comprising:
obtaining, by the one or more processors, a plurality of sample language entities;
with respect to one of the plurality of sample language entities, determining, by the one or more processors, respective sample pinyin codes for respective Chinese characters comprised in the sample language entity;
generating, by the one or more processors, respective sample pinyin features from the respective sample pinyin codes; and
training, by the one or more processors, the mapping based on the respective sample pinyin features and the sample languag

Assignees

• IBM

Inventors

• Jiang Jian Min
• Ni Yuan
• Tang Guo Yu
• Zhao Shiwan
• Xie Guo Tong

Classifications

• G06V30/293

  of characters other than Kanji, Hiragana or Katakana · CPC title

• G06V30/153

  using recognition of characters or words · CPC title

• G06V30/268

  Lexical context · CPC title

• G06F40/279 Primary

  Recognition of textual entities · CPC title

• G06F40/53 Primary

  Processing of non-Latin text (kana-to-kanji conversion G06F40/129; vowelisation G06F40/232) · CPC title