Token Based Dynamic Data Indexing With Integrated Security

Patent metadata

Field	Value
Publication number	US-2024386004-A1
Application number	US-202418419625-A
Country	US
Kind code	A1
Filing date	Jan 23, 2024
Priority date	Jul 17, 2013
Publication date	Nov 21, 2024
Grant date	—

Abstract

Semantic analysis to resolve ambiguous user input data with respect to a request for data includes identifying tokens based on the text string, identifying a tables in a database, wherein a token indicates a column from a table and a token indicates a column from another table, identifying candidate join paths for joining tables, obtaining respective candidate results using the candidate join paths, outputting combined candidate results including values from the respective candidate results, obtaining second user input data indicating a selected value, identifying the request for data as unambiguous and identifying a selected join path based on the selected value, and, in response to identifying the request for data as unambiguous, outputting data responsive to the request for data using the selected join path.

First claim

1 . (canceled)

2 . A method comprising:
obtaining, by an information retrieval system, from a database management system, enterprise data; automatically generating, by the information retrieval system, enterprise data tokens representing the enterprise data; automatically indexing, by the information retrieval system, the enterprise data tokens in an index structure of the information retrieval system; obtaining, by the information retrieval system, first user input data including a natural language string expressing a request for data from the information retrieval system; automatically generating, by the information retrieval system, a semantic representation of the natural language string, in a form that differs from a structured query language of the database management system, wherein generating the semantic representation includes:
traversing the index structure to match a portion of the natural language string to an enterprise data token from the enterprise data tokens;
including the enterprise data token in the semantic representation;
automatically converting, by the information retrieval system, the semantic representation into a structured query language query expressing the request for data; obtaining, by the information retrieval system, from the database management system, in response to the structured query language query, results data responsive to the request for data; and automatically outputting, for presentation to a user, the results data.

3 . The method of claim 1 , wherein indexing the enterprise data tokens includes:
including, in the index structure, a root node; including, in the index structure, a first branch depending from the root node, the first branch representing a first symbol from the enterprise data token; and including, in the index structure, a security bitmask for the enterprise data token, such that a security bitmask for the first branch at the root node is a hierarchical logical disjunction based on the security bitmask for the enterprise data token.

4 . The method of claim 3 , wherein:
obtaining the first user input data includes obtaining a security bitmask for the first user input data; and automatically generating the semantic representation includes:
determining that the first symbol matches a symbol from the portion of the natural language string; and
determining that a horizontal logical disjunction of a vertical logical conjunction of the security bitmask for the first branch and the security bitmask for the first user input data indicates authorization.

5 . The method of claim 1 , wherein:
obtaining the first user input data includes obtaining the first user input data from a user device; and automatically outputting the results data includes outputting the results data to the user device.

6 . The method of claim 1 , wherein:
automatically indexing the enterprise data tokens includes:
including, in the index structure, data indicating an association between the enterprise data token and a first table stored in the database management system; and
including, in the index structure, data indicating an association between a second enterprise data token from the enterprise data tokens and a second table stored in the database management system;
automatically generating the semantic representation includes:
traversing the index structure to match a second portion of the natural language string to the second enterprise data token;
including the second enterprise data token in the semantic representation;
identifying a join path for joining data from the first table with data from the second table;
including data indicating the join path in the semantic representation; and
automatically converting the semantic representation includes:
including data indicating the join path in the structured query language query.

7 . The method of claim 6 , wherein:
obtaining the enterprise data includes:
obtaining relationship data indicating a relationship between the first table and the second table stored; and
storing the relationship data in the information retrieval system; and
identifying the join path includes using the relationship data.

8 . The method of claim 1 , wherein:
automatically indexing the enterprise data tokens includes:
including, in the index structure, data indicating an association between the enterprise data token and a column of a table stored in the database management system; and
automatically generating the semantic representation includes:
in response to determining, by a finite state machine of the information retrieval system, that the column is a measure column, including, in the semantic representation, data indicating an aggregation operation with respect to the measure column.

9 . The method of claim 1 , wherein:
automatically indexing the enterprise data tokens includes:
including, in the index structure, data indicating an association between the enterprise data token and a first column of a first table stored in the database management system;
automatically generating the semantic representation includes:
identifying candidate joint paths, wherein identifying the candidate join paths includes:
identifying a first candidate join path for joining data from the first column with data from a second column from a second table stored in the database management system; and
identifying a second candidate join path for joining data from the first column with data from a third column from the second table;

obtaining second user input data indicating a join path from the candidate join paths;
including data indicating the join path in the semantic representation; and
automatically converting the semantic representation includes:
including data indicating the join path in the structured query language query.

10 . A non-transitory computer-readable storage medium, comprising executable instructions that, when executed by a processor, facilitate performance of operations, comprising:
obtaining, by an information retrieval system, from a database management system, enterprise data; automatically generating, by the information retrieval system, enterprise data tokens representing the enterprise data; automatically indexing, by the information retrieval system, the enterprise data tokens in an index structure of the information retrieval system; obtaining, by the information retrieval system, first user input data including a natural language string expressing a request for data from the information retrieval system; automatically generating, by the information retrieval system, a semantic representation of the natural language string, in a form that differs from a structured query language of the database management system, wherein generating the semantic representation includes:
traversing the index structure to match a portion of the natural language string to an enterprise data token from the enterprise data tokens;
including the enterprise data token in the semantic representation;
automatically converting, by the information retrieval system, the semantic representation into a structured query language query expressing the request for data; obtaining, by the information retrieval system, from the database management system, in response to the structured query language query, results data responsive to the request for data; and automatically outputting, for presentation to a user, the results data.

11 . The non-transitory computer-readable storage medium of claim 10 , wherein indexing the enterprise data tokens includes:
including, in the index structure, a root node; including, in the index structure, a first branch depending from the

Assignees

• Thoughtspot Inc

Inventors

• Gupta Shashank
• Prakash Amit
• Singh Ajeet
• Agarwal Shikhar
• Chauhan Shailesh
• Shekhar Satyam

Classifications

• G06N5/04

  Inference or reasoning models · CPC title

• G06F16/3329

  Natural language query formulation · CPC title

• G06F16/24522

  Translation of natural language queries to structured queries · CPC title

• G06F16/243

  Natural language query formulation · CPC title

• G06F16/3334

  Selection or weighting of terms from queries, including natural language queries · CPC title