Automatic accuracy estimation for audio transcriptions

Patent metadata

Field	Value
Publication number	US-9892725-B2
Application number	US-201715398779-A
Country	US
Kind code	B2
Filing date	Jan 5, 2017
Priority date	May 3, 2012
Publication date	Feb 13, 2018
Grant date	Feb 13, 2018

Abstract

Embodiments of the present invention provide an approach for estimating the accuracy of a transcription of a voice recording. Specifically, in a typical embodiment, each word of a transcription of a voice recording is checked against a customer-specific dictionary and/or a common language dictionary. The number of words not found in either dictionary is determined. An accuracy number for the transcription is calculated from the number of said words not found and the total number of words in the transcription.

First claim

What is claimed is:

1. A method of assigning a confidence level to at least one axiom extracted from a text, comprising:
determining a number of accurate words in text data based on a spell check function, wherein the text data are extracted from different sources;
dividing a number of accurate words from the text data by a total number of words in the text data;
assigning a greater weight to at least one word exceeding a predetermined number of characters as compared to at least one other word below the predetermined number of characters;
automatically extracting, using natural language processing, from a knowledge base stored in a computer infrastructure, the at least one axiom, wherein the at least one axiom is associated with at least one word from the text data, and wherein the axiom comprises a computer-parsable definition of a relationship of data to at least one of the words in the text data;
assigning a confidence level to the at least one axiom based on a result of the dividing and the assigning, wherein the confidence level is assigned based on an output of a Gaussian function applied to the result of the dividing and the assigning;
receiving a query from a user device, wherein the query includes at least one word;
matching the at least one word with the at least one axiom using the knowledge base; and
providing the at least one axiom from the knowledge base to the user device based on the matching and the assigned confidence level.

2. The method of claim 1 , further comprising:
comparing the text data to the data structure, wherein the data structure comprises a dictionary.

3. The method of claim 2 , further comprising:
determining a number of inaccurate words in the text data based on the comparing.

4. The method of claim 2 , wherein the dictionary comprises a customer specific dictionary.

5. The method of claim 2 , wherein the dictionary comprises a dictionary of common language words.

6. The method of claim 2 , wherein the determining the number of inaccurate words comprises identifying a number of words not found in the dictionary.

7. The method of claim 1 , wherein a solution service provider provides a computer infrastructure operable to perform the method.

8. A system for assigning a confidence level to at least one axiom extracted from a text, comprising:
a memory medium comprising instructions;
a bus coupled to the memory medium; and
an audio transcription tool coupled to the bus that when executing the instructions causes the system to:
determine a number of accurate words in text data based on a spell check function, wherein the text data are extracted from different sources;
divide a number of accurate words from the text by a total number of words in the text data;
assign a greater weight to at least one word exceeding a predetermined number of characters as compared to at least one other word below the predetermined number of characters;
automatically extract, using natural language processing, from a knowledge base stored in a computer infrastructure, the at least one axiom, wherein the at least one axiom is associated with at least one word from the text data, and wherein the axiom comprises a computer-parsable definition of a relationship of data to at least one of the words in the text data;
assign a confidence level to the at least one axiom based on a result of the dividing and the assigning, wherein the confidence level is assigned based on an output of a Gaussian function applied to the result of the dividing and the assigning;
receive a query from a user device, wherein the query includes at least one word;
match the at least one word with the at least one axiom using the knowledge base; and
provide the at least one axiom from the knowledge base to the user device based on the matching and the assigned confidence level.

9. The system of claim 8 , the an audio transcription tool coupled to the bus that when executing the instructions further causes the system to:
compare the text data to the data structure, wherein the data structure comprises a dictionary; and
determine a number of inaccurate words in the text data.

10. The system of claim 9 , wherein the dictionary comprises a customer specific dictionary.

11. The system of claim 9 , wherein the dictionary comprises a dictionary of common language words.

12. The system of claim 9 , wherein the determining the number of inaccurate words comprises identifying a number of words not found in the dictionary.

13. A computer program product for assigning a confidence level to at least one axiom extracted from a text, the computer program product comprising a computer readable hardware storage device, and program instructions stored on the computer readable hardware storage device, to:
determine a number of accurate words in text data based on a spell check function, wherein the text data are extracted from different sources;
divide a number of accurate words from the text data by a total number of words in the text;
assign a greater weight to at least one word exceeding a predetermined number of characters as compared to at least one other word below the predetermined number of characters;
automatically extract, using natural language processing, from a knowledge base stored in a computer infrastructure, the at least one axiom, wherein the at least one axiom is associated with at least one word from the text data, and wherein the axiom comprises a computer-parsable definition of a relationship of data to at least one of the words in the text data;
assign a confidence level to the at least one axiom based on a result of the dividing and the assigning, wherein the confidence level is assigned based on an output of a Gaussian function applied to the result of the dividing and the assigning;
receive a query from a user device, wherein the query includes at least one word;
match the at least one word with the at least one axiom using the knowledge base; and
provide the at least one axiom from the knowledge base to the user device based on the matching and the assigned confidence level.

14. The computer program product of claim 13 , the computer program product comprising a computer readable hardware storage device, and program instructions stored on the computer readable storage medium, to:
compare the text data to the data structure, wherein the data structure comprises a dictionary; and
determine a number of inaccurate words in the text data.

15. The computer program product of claim 14 , wherein the dictionary comprises a customer specific dictionary.

16. The computer program product of claim 14 , wherein the dictionary comprises a dictionary of common language words.

17. The computer program product of claim 14 , wherein the determining the number of inaccurate words comprises identifying a number of words not found in the dictionary.

Assignees

• IBM

Inventors

• Bostick James E
• Ganci Jr John M
• Kaemmerer John P
• Trim Craig M

Classifications

• G10L15/01 Primary

  Assessment or evaluation of speech recognition systems · CPC title

• G06F40/189

  Automatic justification · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G06F40/242

  Dictionaries · CPC title

• G06F40/51

  Translation evaluation · CPC title