Language model training method and device

Patent metadata

Field	Value
Publication number	US-2017125013-A1
Application number	US-201615242065-A
Country	US
Kind code	A1
Filing date	Aug 19, 2016
Priority date	Oct 29, 2015
Publication date	May 4, 2017
Grant date	—

Abstract

The present disclosure provides a language model training method and device, including: obtaining a universal language model in an offline training mode, and clipping the universal language model to obtain a clipped language model; obtaining a log language model of logs within a preset time period in an online training mode; fusing the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and fusing the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding. The method is used for solving the problem that a language model obtained offline in the prior art has poor coverage on new corpora, resulting in a reduced language recognition rate.

First claim

What is claimed is:

1 . A language model training method, comprising:
obtaining a universal language model in an offline training mode, and clipping the universal language model to obtain a clipped language model; obtaining a log language model of logs within a preset time period in an online training mode; fusing the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and fusing the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding.

2 . The method of claim 1 , wherein the obtaining a log language model of logs within a preset time period in an online training mode comprises:
obtaining log information within the preset time period, filtering the log information, and carrying out word segmentation processing on the filtered log information to obtain a log model training corpus within the preset time period; and training the log model training corpus to obtain the log language model.

3 . The method of claim 1 , wherein the clipped language model is a tri-gram language model, and correspondingly, the first fusion language model is a tri-gram fusion language model; and
the universal language model is a tetra-gram language model, and correspondingly, the second fusion language model is a tetra-gram fusion language model.

4 . The method of any one of claim 1 , wherein the obtaining a universal language model in an offline training mode comprises:
collecting a model training corpus of each field; for each field, training the model training corpus of the field to obtain the language model of the field; and generating the collected language models corresponding to all fields into the universal language model in the interpolation mode.

5 . The method of claim 4 , wherein the clipping the universal language model to obtain a clipped language model comprises:
clipping the universal language model in a language model clipping mode based on entropy to obtain a second language model LM2; clipping the second language model LM2 in the language model clipping mode based on entropy to obtain a third language model LM3; and extracting the tri-gram language model from the third language model LM3, and clipping the extracted tri-gram language model to obtain the clipped language model LM4.

6 . An electronic device, comprising:
at least one processor; and a memory communicably connected with the at least one processor for storing instructions executable by the at least one processor, wherein execution of the instructions by the at least one processor causes the at least one processor to: obtain a universal language model in an offline training mode; clip the universal language model to obtain a clipped language model; obtain a log language model of logs within a preset time period in an online training mode; fuse the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and fuse the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding.

7 . The device of claim 6 , wherein the processor is further configured to perform the following steps:
obtaining log information within the preset time period, filtering the log information, and carrying out word segmentation processing on the filtered log information to obtain a log model training corpus within the preset time period; and training the log model training corpus to obtain the log language model.

8 . The device of claim 6 , wherein the clipped language model is a tri-gram language model, and correspondingly, the first fusion language model is a tri-gram fusion language model; and
the universal language model is a tetra-gram language model, and correspondingly, the second fusion language model is a tetra-gram fusion language model.

9 . The device of claim 6 , wherein the processor is further configured to perform the following steps:
collecting a model training corpus of each field; for each field, training the model training corpus of the field to obtain the language model of the field; and generating the collected language models corresponding to all fields into the universal language model in the interpolation mode.

10 . The device of claim 9 , wherein the processor is further configured to perform the following steps:
clipping the universal language model in a language model clipping mode based on entropy to obtain a second language model LM2; clipping the second language model LM2 in the language model clipping mode based on entropy to obtain a third language model LM3; and extracting the tri-gram language model from the third language model LM3, and clipping the extracted tri-gram language model to obtain the clipped language model LM4.

11 . A non-transitory computer-readable storage medium storing executable instructions that, when executed by an electronic device with a touch-sensitive display, cause the electronic device to:
obtain a universal language model in an offline training mode; clip the universal language model to obtain a clipped language model; obtain a log language model of logs within a preset time period in an online training mode; fuse the clipped language model with the log language model to obtain a first fusion language model used for carrying out first time decoding; and fuse the universal language model with the log language model to obtain a second fusion language model used for carrying out second time decoding.

12 . The non-transitory computer-readable storage medium of claim 11 , wherein the electronic device is further configured to perform the following steps:
obtaining log information within the preset time period, filtering the log information, and carrying out word segmentation processing on the filtered log information to obtain a log model training corpus within the preset time period; and training the log model training corpus to obtain the log language model.

13 . The non-transitory computer-readable storage medium of claim 11 , wherein the clipped language model is a tri-gram language model, and correspondingly, the first fusion language model is a tri-gram fusion language model; and
the universal language model is a tetra-gram language model, and correspondingly, the second fusion language model is a tetra-gram fusion language model.

14 . The non-transitory computer-readable storage medium of claim 11 , wherein the electronic device is further configured to perform the following steps:
collecting a model training corpus of each field; for each field, training the model training corpus of the field to obtain the language model of the field; and generating the collected language models corresponding to all fields into the universal language model in the interpolation mode.

15 . The non-transitory computer-readable storage medium of claim 14 , wherein the electronic device is further configured to perform the following steps:
clipping the universal language model in a language model clipping mode based on entropy to obtain a second language model LM2; clipping the second language model LM2 in the language model clipping mode based on entropy to obtain a third language model LM3; and extracting the tri-gram language model from the third language model LM3, and clipping the extracted tri-gram language model to obtain the clipped language model LM4.

Assignees

• Le Holdings Beijing Co Ltd
• Le Shi Zhi Xin Electronic Tech (Tianjin) Ltd

Inventors

• Yan Zhiyong

Classifications

• G10L2015/0633

  using lexical or orthographic knowledge sources · CPC title

• G10L15/063 Primary

  Training · CPC title

• G10L2015/0635

  updating or merging of old and new templates; Mean values; Weighting · CPC title

• G10L15/183

  using context dependencies, e.g. language models · CPC title

• G10L15/197

  Probabilistic grammars, e.g. word n-grams · CPC title