Search and recommendation engine allowing recommendation-aware placement of data assets to minimize maximal load

Patent metadata

Field	Value
Publication number	US-12561384-B2
Application number	US-202217875750-A
Country	US
Kind code	B2
Filing date	Jul 28, 2022
Priority date	Mar 29, 2016
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

A search engine responding to a user query to find relevant data assets in a federation business data lake (FBDL) system based on interactions of known users interacting with data assets in the FBDL system. Data assets are optimally placed for minimal latency or maximal load. Data asset recommendations and past data asset access information are input as features to a time-series model for predicting future data access patterns. An expected latency and load risk is then determined and scored by a weighted mean of these values, and placement optimization is simulated using an optimization method (e.g., genetic algorithm). Using the scoring and simulation, a data asset placement engine is then used to move the locations of the data assets to minimize maximal load that comprises a load risk representing how close a current load is to a service level agreement (SLA) requirement set by a system provider.

First claim

What is claimed is:

1 . A computer-implemented method of optimizing placement of data assets in a data retrieval system storing data assets for users in an enterprise, comprising:
generating search results and data asset recommendations in response to a target user query; providing the recommendations and past data asset access information for data assets as features to a time-series model for predicting future data access patterns by the target user; determining and scoring an expected latency and load risk created by a weighted mean of these expected latency and load risk values; simulating a placement optimization using an optimization method; and moving the data assets from one location to another location in the system using the scoring and simulation so as to minimize maximal load, wherein the data processing system is maintained by a large scale enterprise, and wherein the data assets comprise Big Data-scale data sets, and wherein the data assets comprise databases, stacks of databases, file systems, and enterprise services, and wherein the data assets are accessed through a Hadoop layer storing open source software components to control storing, processing, and analyzing the data, and wherein the data assets are stored in storage devices organized into arrays, and the arrays are located in one or more data centers of a federation business data lake (FBDL) storage system.

2 . The method of claim 1 wherein the maximal load comprises a load risk representing how close a current load is to a service level agreement (SLA) requirement set by a system provider.

3 . The method of claim 2 wherein the data retrieval system comprises a search engine processing the query from the target user, the search engine returning one or more data asset recommendations responsive to the query.

4 . The method of claim 3 further comprising:
monitoring and recording, by a monitoring component of the server, all interactions of a plurality of known users, including a first user and a target user, each interaction comprising an activity that triggers a read/write cycle to the storage;
first deriving a similarity of each of the plurality of known users to the target user based on respective past and current data retrieval patterns of each of known users for data queried in the search engine;
modeling a plurality of possible users for whom there are no known interactions with the plurality of known users or the data assets to constitute missing features;
training respective a generative model using different random seeds for an inference engine through reconstructive self-supervised learning (SSL) techniques to generate possible values for the missing features; and
providing the generated possible values to a consensus mechanism to generate an integrated recommendation to the target user.

5 . The method of claim 4 wherein the partial features comprise past searches, user interactions with the database, user interactions with each other, user profiles, product/service characteristics, user access patterns between users and the FBDL, user profiles and interactions among users.

6 . The method of claim 5 wherein the optimization method comprises a genetic algorithm (GA).

7 . The method of claim 1 wherein the moving step moves data assets from one disk to another disk within a storage device, or from one array to another array, or from one data center to another data center.

8 . A method of processing queries input to a data retrieval system storing data assets for users in an enterprise, comprising:
storing, in a federation business data lake (FBDL) storage maintained for a large-scale data processing system, data assets retrievable by a user in one or more possible locations along a scale of disks within a storage device, storage devices within arrays, and arrays within data centers; providing a search engine for entry of queries by a target user looking for data assets in the FBDL storage; generating search results and data asset recommendations in response to a target user query input to the search engine; providing the recommendations and past data asset access information for data assets as features to a time-series model for predicting future data access patterns by the target user; simulating an optimized placement of the data assets to maximal load using an optimization method; and moving the data assets from one location to another location in the system using the scoring and simulation.

9 . The method of claim 8 wherein the maximal load comprises a load risk representing how close a current load is to a service level agreement (SLA) requirement set by a system provider.

10 . The method of claim 9 further comprising determining and scoring an expected latency and load risk created by a weighted mean of these expected latency and load risk values.

11 . The method of claim 9 wherein the data processing system is maintained by a large scale enterprise, and wherein the data assets comprise Big Data-scale data sets, and wherein the data assets comprise databases, stacks of databases, file systems, and enterprise services, and wherein the data assets are accessed through a Hadoop layer storing open source software components to control storing, processing, and analyzing the data.

12 . The method of claim 11 wherein the moving step moves data assets from one disk to another disk within a storage device, or from one array to another array, or from one data center to another data center.

13 . The method of claim 11 further comprising:
monitoring and recording, by a monitoring component of the server, all interactions of a plurality of known users, including a first user and a target user, each interaction comprising an activity that triggers a read/write cycle to the storage;
first deriving a similarity of each of the plurality of known users to the target user based on respective past and current data retrieval patterns of each of known users for data queried in the search engine;
modeling a plurality of possible users for whom there are no known interactions with the plurality of known users or the data assets to constitute missing features;
training respective a generative model using different random seeds for an inference engine through reconstructive self-supervised learning (SSL) techniques to generate possible values for the missing features; and
providing the generated possible values to a consensus mechanism to generate an integrated recommendation to the target user.

14 . The method of claim 13 wherein the partial features comprise past searches, user interactions with the database, user interactions with each other, user profiles, product/service characteristics, user access patterns between users and the FBDL storage, user profiles and interactions among users.

15 . A computer network system for optimizing placement of data assets in a data retrieval system storing data assets for users in an enterprise, comprising:
a search engine generating search results and data asset recommendations in response to a target user query; a time-series modeling component receiving the recommendations and past data asset access information for data assets as features, and configured to predict future data access patterns by the target user; and a data asset placement engine receiving the future data asset access pattern prediction and determining and scoring an expected latency and load risk created by a weighted mean of these expected latency and load risk values, the data asset placement engine further simulating a placement optimization using an optimization method, and moving the da

Assignees

• Dell Products Lp
• Emc Ip Holding Co Llc

Inventors

• Savir Amihai
• Ezrielev Ofir
• Harush Oshry Ben

Classifications

• G06F16/9535 Primary

  Search customisation based on user profiles and personalisation · CPC title