Method and system for generating training data for classifying intents in a conversational system

Patent metadata

Field	Value
Publication number	US-12405976-B2
Application number	US-202218062632-A
Country	US
Kind code	B2
Filing date	Dec 7, 2022
Priority date	Dec 16, 2021
Publication date	Sep 2, 2025
Grant date	Sep 2, 2025

Abstract

Embodiments of present disclosure relates to method training data generation system for generating training data for classifying intent in conversational system. The training data generation system receives database schema and creates SQL/NoSQL queries. The training data generation system generates natural language queries for the SQL/NoSQL queries. Further, the training data generation system generates training data for intents associated with the natural language queries and provides to classification models associated with conversational system for classification of intents. Embodiments of present disclosure relates to method and conversational system for providing natural language response for query. The conversational system receives query from user and classifies intent of the query and provides relevant response by mapping the query with the SQL/NoSQL queries generated by the training data generation system. Thus, the present disclosure generates conversational system without manually providing training data for classifying intents in real-time.

First claim

We claim:

1. A method for providing natural language response to a user for a query using a conversational system ( 104 1 ), the method comprising:
receiving, by a conversational system ( 104 1 ), a query in a natural language from a user;
classifying, by the conversational system ( 104 1 ), an intent associated with the query by using a classification model associated with the conversational system ( 104 1 ), wherein the classification model is trained by using a method for generating training data for classifying intents in a conversational system ( 104 1 ), the generating method comprising:
receiving, by a training data generation system ( 101 ), structured databases schema, wherein the structured databases schema comprise information related to an enterprise;
creating, by the generation system ( 101 ), at least one of one or more Structured Query Language (SQL) and not only SQL (NoSQL) queries based on the structured databases schema by using predefined rules;
converting, by the generation system ( 101 ), at least one of the one or more SQL and NoSQL queries into respective one or more natural language queries using a Deep Learning Neural Network (DNN) model, wherein the DNN model is trained using a first knowledge corpus ( 210 ) of a specific natural language;
creating, by the generation system ( 101 ), a second knowledge corpus based on the one or more natural language queries and the first knowledge corpus ( 210 ) using semantic analysis methodology; and
generating, by the generation system ( 101 ), training data for intents associated with each of the one or more natural language queries using the second knowledge corpus, wherein the training data is provided to one or more classification models for classifying an intent in the conversational system ( 104 1 );

mapping, by the conversational system ( 104 1 ), the query with at least one of, one or more SQL and NoSQL queries prestored in a database using the classified intent; and
providing, by the conversational system ( 104 1 ), a natural language response to the user from the database based on the mapped at least one of the one or more SQL and NoSQL queries.

2. The method as claimed in claim 1 , wherein the creation of at least one of the one or more SQL and NoSQL queries based on the structured databases schema, comprises:
extracting, by the generation system ( 101 ), details of a plurality of columns from each table of the structured databases schema and identifying data type corresponding to each of the plurality of columns; and
creating, by the generation system ( 101 ), at least one of the one or more SQL and NoSQL queries for each of the plurality of columns using the predefined rules associated with respective data type.

3. The method as claimed in claim 1 , wherein converting at least one of the one or more SQL and NoSQL queries into the respective one or more natural language queries, comprises:
obtaining, by the generation system ( 101 ), information related to at least one of the one or more SQL and NoSQL queries, wherein the information comprises types of operations used in at least one of the one or more SQL and NoSQL queries, column names of each table used in at least one of the one or more SQL and NoSQL queries, and data type of each column name;
normalising, by the generation system ( 101 ), the extracted information using at least one of stemming, spelling correction and abbreviation expansion; and
generating, by the generation system ( 101 ), the one or more natural language queries by assigning weights and bias to the normalized at least one of the one or more SQL and No SQL queries using the DNN model and the first knowledge corpus ( 210 ).

4. The method as claimed in claim 1 , wherein generating the training data for the intents associated with each of the one or more natural language queries, comprises:
tagging, by the generation system ( 101 ), the second knowledge corpus into one or more dialog tags using N-gram and topic modelling;
clustering, by the generation system ( 101 ), the one or more dialog tags based on predefined parameters and labelling the clustered one or more dialog tags; and
generating, by the generation system ( 101 ), the training data for the intents associated with each of the one or more natural language queries based on the clustered and the labelled one or more dialog tags, wherein the intents comprise tags, patterns, responses, and context associated with the one or more natural language queries.

5. The method as claimed in claim 1 , wherein the query includes one of a text message, and a voice message.

6. A conversational system ( 104 1 ) for providing natural language response to a user for a query, comprising:
a training data generation system ( 101 ) for generating training data for classifying intents in a conversational system ( 104 1 ), comprising:
a training data processor ( 105 ); and
a training data memory ( 107 ) communicatively coupled to the training data processor ( 105 ), wherein the training data memory ( 107 ) stores processor-executable instructions, which, on execution, cause the training data processor ( 105 ) to:
receive structured databases schema, wherein the structured databases schema comprise information related to an enterprise;
create at least one of one or more Structured Query Language (SQL) and not only SQL (NoSQL) queries based on the structured databases schema by using predefined rules;
convert at least one of the one or more SQL and NoSQL queries into respective one or more natural language queries using a Deep Learning Neural Network (DNN) model, wherein the DNN model is trained using a first knowledge corpus ( 210 ) of a specific natural language;
create a second knowledge corpus based on the one or more natural language queries and the first knowledge corpus ( 210 ) using semantic analysis methodology; and
generate training data for intents associated with each of the one or more natural language queries using the second knowledge corpus, wherein the training data is provided to one or more classification models for classifying an intent in the conversational system ( 104 1 );

a processor ( 110 ); and
a memory ( 112 ) communicatively coupled to the processor ( 110 ), wherein the memory ( 112 ) stores processor-executable instructions, which, on execution, cause the processor ( 110 ) to:
receive a query in a natural language from a user;
classify an intent associated with the query by using a classification model associated with the conversational system ( 104 1 ), wherein the classification model is trained by using the training data generated by the training data generation system ( 101 );
map the query with at least one of one or more SQL and NoSQL queries prestored in a database using the classified intent; and
provide a natural language response to the user from the database based on the mapped at least one of the one or more SQL and NoSQL queries.

7. The conversational system ( 104 1 ) as claimed in claim 6 , wherein the training data processor ( 105 ) creates at least one of the one or more SQL and NoSQL queries by:
extracting details of a plurality of columns from each table of the structured databases schema and identifying data type corresponding to each of the plurality of columns; and
creating at least one of the one or more SQL and NoSQL queries for each of the plurality of columns using the predefined rules associated with respective data type.

8. The conversational system ( 104 1 ) training data generation system ( 101 ) as claimed in claim 6 , wherein the training data processor ( 105 ) converts at least one of the one or more SQL and NoSQL queries into the respective one or more natural language queries by:
obtaining informatio

Assignees

• Hitachi Ltd

Inventors

• Mustafi Joy
• Telang Geet Tapan
• Samon Thiruvengadam
• Banerjee Kingshuk

Classifications

• G06F40/30

  Semantic analysis · CPC title

• G06F40/40

  Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30) · CPC title

• G06F40/35

  Discourse or dialogue representation · CPC title

• G06F40/216

  using statistical methods · CPC title

• G06F16/3329 Primary

  Natural language query formulation · CPC title