Management of indexed data to improve content retrieval processing

Patent metadata

Field	Value
Publication number	US-12353382-B2
Application number	US-202218062991-A
Country	US
Kind code	B2
Filing date	Dec 7, 2022
Priority date	Dec 19, 2019
Publication date	Jul 8, 2025
Grant date	Jul 8, 2025

Abstract

The present disclosure relates to processing operations configured to uniquely utilize indexing of content to improve content retrieval processing, particularly when working with large data sets. The techniques described herein enables efficient content retrieval when working with large data sets such as those that may be associated with a plurality of tenants of a data storage application/service. Among other technical advantages, the present disclosure is applicable to train a classifier using relevant samples based on text search in tenant-specific scenarios, where accurate searching can be executed for content associated with one or more tenant accounts of an application/service concurrently in milliseconds even in instances where there may be millions of documents to be searched. As an example, exemplary data shards may be generated and managed for efficient and scalable content retrieval processing including training of a classifier (e.g., artificial intelligence classifier) and real-time (or near real-time) query processing.

First claim

What is claimed is:

1. A method, comprising:
pre-loading, into a memory, data shards comprising a plurality of indexes from indexing file content representative of a randomized sampling of the file content;
receiving a search query for searching the file content from a user via a user interface of a cloud-based service;
processing the search query with a trained artificial intelligence classifier trained to search indexing across the data shards to retrieve content responsive to the search query;
reading, from the memory, the data shards during the processing the search query by the trained artificial intelligence classifier; and
providing the content responsive to the search query.

2. The method of claim 1 , further comprising:
detecting the user is accessing the cloud-based service, wherein the pre-loading the data shards is in response to the detecting.

3. The method of claim 1 , further comprising:
generating the data shards.

4. The method of claim 3 , wherein the generating the data shards comprises:
identifying a predetermined number of files for a size of each of the data shards, and randomly selecting, as the randomized sampling, indexes associated with the predetermined number of files from the file content.

5. The method of claim 1 , wherein:
the cloud-based service is a multi-tenant cloud-based service;
the user is a tenant of the multi-tenant cloud-based service; and
the data shards are data shards containing data specific to the tenant.

6. The method of claim 1 , further comprising:
generating a processing queue that groups the data shards for processing during content retrieval.

7. The method of claim 6 , wherein:
the cloud-based service is a multi-tenant cloud-based service; and
the processing queue groups the data shards into tenant-specific data shards.

8. The method of claim 1 , further comprising:
determining a number of data shards to be processed concurrently during rounds of content retrieval processing; and
load balancing by determining computing resources to allocate based on the number of data shards for a given round of content retrieval.

9. The method of claim 1 , wherein the reading the data shards comprises reading the data shards in a randomized order.

10. A system, comprising:
one or more processors; and
a memory having stored thereon instructions that, upon execution by the one or more processors, cause the one or more processors to:
pre-load, into the memory, data shards comprising a plurality of indexes from indexing file content representative of a randomized sampling of the file content;
receive a search query for searching the file content from a user via a user interface of a cloud-based service;
process the search query with a trained artificial intelligence classifier trained to search indexing across the data shards to retrieve content responsive to the search query;
read, from the memory, the data shards during the processing the search query by the trained artificial intelligence classifier; and
providing the content responsive to the search query.

11. The system of claim 10 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
detect the user is accessing the cloud-based service, wherein the pre-loading the data shards is in response to the detecting.

12. The system of claim 10 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
generate the data shards.

13. The system of claim 12 , wherein the instructions to generate the data shards comprises further instructions that, upon execution by the one or more processors, cause the one or more processors to:
identify a predetermined number of files for a size of each of the data shards, and randomly selecting, as the randomized sampling, indexes associated with the predetermined number of files from the file content.

14. The system of claim 10 , wherein:
the cloud-based service is a multi-tenant cloud-based service;
the user is a tenant of the multi-tenant cloud-based service; and
the data shards are data shards containing data specific to the tenant.

15. The system of claim 10 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
generate a processing queue that groups the data shards for processing during content retrieval.

16. The system of claim 15 , wherein:
the cloud-based service is a multi-tenant cloud-based service; and
the processing queue groups the data shards into tenant-specific data shards.

17. The system of claim 10 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
determine a number of data shards to be processed concurrently during rounds of content retrieval processing; and
load balance by determining computing resources to allocate based on the number of data shards for a given round of content retrieval.

18. The system of claim 10 , wherein the reading the data shards comprises reading the data shards in a randomized order.

19. A computer-readable memory device having stored thereon instructions that, upon execution by one or more processors, cause the one or more processors to:
pre-load, into a memory, data shards comprising a plurality of indexes from indexing file content representative of a randomized sampling of the file content;
receive a search query for searching the file content from a user via a user interface of a cloud-based service;
process the search query with a trained artificial intelligence classifier trained to search indexing across the data shards to retrieve content responsive to the search query;
read, from the memory, the data shards during the processing the search query by the trained artificial intelligence classifier; and
providing the content responsive to the search query.

20. The computer-readable memory device of claim 19 , wherein the instructions comprise further instructions that, upon execution by the one or more processors, cause the one or more processors to:
detect the user is executing the service, wherein the pre-loading the data shards is in response to the detecting.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Deshpande Saurabh Sanjay
• Mikhail Mina
• Hurst Matthew Francis
• Mansour Riham Hassan Abdel-Moneim

Classifications

• G06F18/2148

  characterised by the process organisation or structure, e.g. boosting cascade · CPC title

• G06F16/22

  Indexing; Data structures therefor; Storage structures · CPC title

• G06F16/24549

  Run-time optimisation · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F16/2272

  Management thereof · CPC title