Method for selecting annotated sample, apparatus, electronic device and storage medium

Patent metadata

Field	Value
Publication number	US-11907668-B2
Application number	US-202218148904-A
Country	US
Kind code	B2
Filing date	Dec 30, 2022
Priority date	Feb 9, 2022
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

The present disclosure provides a method for selecting an annotated sample. The method includes: determining a first attribute and a second attribute of a sample characteristic; in which the first attribute is a characteristic attribute of the sample characteristic in a source field sample set, and the second attribute is a characteristic attribute of the sample characteristic in a target field sample set; and determining a target annotated sample from a plurality of candidate annotated samples of the source field sample set according to the first attribute and the second attribute; in which the target annotated sample is configured to train a classification model, the classification model includes a model for determining an emotion polarity by analyzing an input sample to be classified.

First claim

What is claimed is:

1. A method for selecting an annotated sample, comprising:
determining a first attribute and a second attribute of a sample characteristic; wherein the first attribute is a characteristic attribute of the sample characteristic in a source field sample set, and the second attribute is a characteristic attribute of the sample characteristic in a target field sample set; and
determining a target annotated sample from a plurality of candidate annotated samples of the source field sample set according to the first attribute and the second attribute; wherein the target annotated sample is configured to train a classification model, the classification model comprises a model for determining an emotion polarity by analyzing an input sample to be classified;
wherein determining the first attribute comprises:
determining a first frequency value of the sample characteristic in the source field sample set;
determining an importance level and an emotion polarity of the sample characteristic in the source field sample set using the first frequency value; and
determining the importance level and the emotion polarity of the sample characteristic in the source field sample set as the first attribute.

2. The method as claimed in claim 1 , wherein determining the importance level of the sample characteristic in the source field sample set using the first frequency value comprises:
determining a first chi-square value of the sample characteristic using the first frequency value and a sample number of the source field sample set; wherein the first chi-square value is a chi-square value of the sample characteristic in the source field sample set; and
determining the importance level of the sample characteristic in the source field sample set based on the first chi-square value.

3. The method as claimed in claim 2 , wherein determining the emotion polarity of the sample characteristic in the source field sample set using the first frequency value comprises:
determining an emotion polarity value of the sample characteristic using the first frequency value and the sample number of the source field sample set; and
determining the emotion polarity of the sample characteristic in the source field sample set using the emotion polarity value.

4. The method as claimed in claim 1 , wherein determining the second attribute comprises:
adding reference annotations for the target field sample set according to a preset condition; wherein the reference annotations comprises a positive annotation or a negative annotation; and
determining the second attribute of the sample characteristic using the reference annotations.

5. The method as claimed in claim 4 , wherein determining the second attribute of the sample characteristic using the reference annotation comprises:
determining a second frequency value of the sample characteristic in the target field sample set using the reference annotations;
determining a reference importance level and a reference emotion polarity of the sample characteristic in the target field sample set using a sample number of the target field sample set and the second frequency value; and
determining the reference importance level and the reference emotion polarity as the second attribute.

6. The method as claimed in claim 5 , wherein determining the reference importance level comprises:
determining a second chi-square value and an estimated deviation value of the sample characteristic using the second frequency value and the sample number of the target field sample set; wherein the second chi-square value is a chi-square value of the sample characteristic in the target field sample set; and
determining the reference importance level of the sample characteristic in the target field sample set using the second chi-square value and the estimated deviation value.

7. The method as claimed in claim 1 , wherein determining the target annotated sample from the plurality of candidate annotated samples of the source field sample set according to the first attribute and the second attribute comprises:
determining a target sample characteristic according to the first attribute and the second attribute; and
determining the target annotated sample by performing data filtration in the source field sample set using the target sample characteristic.

8. The method as claimed in claim 7 , wherein determining the target sample characteristic according to the first attribute and the second attribute comprises:
selecting, in the source field sample set, at least one sample characteristic with both an importance level in the first attribute and a reference importance level in the second attribute meeting a predetermined condition as at least one candidate sample characteristic; and
selecting, from the at least one candidate sample characteristic, a sample characteristic with an emotion polarity in the first attribute being same as a reference emotion polarity in the second attribute as the target sample characteristic.

9. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein,
the memory stores instructions executable by the at least one processor, the instructions are executed by the at least one processor to cause the at least one processor to implement a method for selecting an annotated sample, comprising:
determining a first attribute and a second attribute of a sample characteristic; wherein the first attribute is a characteristic attribute of the sample characteristic in a source field sample set, and the second attribute is a characteristic attribute of the sample characteristic in a target field sample set; and
determining a target annotated sample from a plurality of candidate annotated samples of the source field sample set according to the first attribute and the second attribute; wherein the target annotated sample is configured to train a classification model, the classification model comprises a model for determining an emotion polarity by analyzing an input sample to be classified;
wherein determining the first attribute comprises:
determining a first frequency value of the sample characteristic in the source field sample set;
determining an importance level and an emotion polarity of the sample characteristic in the source field sample set using the first frequency value; and
determining the importance level and the emotion polarity of the sample characteristic in the source field sample set as the first attribute.

10. The electronic device as claimed in claim 9 , wherein determining the importance level of the sample characteristic in the source field sample set using the first frequency value comprises:
determining a first chi-square value of the sample characteristic using the first frequency value and a sample number of the source field sample set; wherein the first chi-square value is a chi-square value of the sample characteristic in the source field sample set; and
determining the importance level of the sample characteristic in the source field sample set based on the first chi-square value.

11. The electronic device as claimed in claim 10 , wherein determining the emotion polarity of the sample characteristic in the source field sample set using the first frequency value comprises:
determining an emotion polarity value of the sample characteristic using the first frequency value and the sample number of the source field sample set; and
determining the emotion polarity of the sample characteristic in the source field sample set using the emotion polarity value.

12. The electronic device as claimed in claim 9 , wherein determining the second attri

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Peng Minlong
• Sun Mingming
• Li Ping

Classifications

• G06F40/30 Primary

  Semantic analysis · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06F16/35 Primary

  Clustering; Classification · CPC title