Hybrid allocation scheme for load-balanced data groups in distributed nodes

Patent metadata

Field	Value
Publication number	US-2026064480-A1
Application number	US-202519368618-A
Country	US
Kind code	A1
Filing date	Oct 24, 2025
Priority date	Sep 29, 2025
Publication date	Mar 5, 2026
Grant date	—

Abstract

Various examples, systems, and methods are disclosed relating to a hybrid allocation scheme for load-balanced data groups in distributed nodes. Some systems can allocate, using an allocation scheme corresponding to a first phase, a plurality of data groups to a plurality of nodes. Some systems can update, using a correction scheme corresponding to a second phase, at least one allocation of at least one data group of the plurality of data groups based at least on a convergence parameter and a resource metric. The resource metric corresponds to at least one resource indicator of a hardware configuration detected from at least one node of the plurality of nodes based at least on performance of at least one node command.

First claim

What is claimed is:

1 . A method, comprising:
allocating, using an allocation scheme corresponding to a first phase, a plurality of data groups to a plurality of nodes; and updating, using a correction scheme corresponding to a second phase, at least one allocation of at least one data group of the plurality of data groups based at least on a convergence parameter and a resource metric, the resource metric corresponding to at least one resource indicator of a hardware configuration detected from at least one node of the plurality of nodes based at least on performance of at least one node command; wherein the method is performed using one or more processors.

2 . The method of claim 1 , wherein to update the at least one allocation of the at least one data group comprises:
determining a resource indicator for each of the plurality of nodes based at least on the hardware configuration corresponding to at least one of a memory usage, a processor usage, a network usage, an application value, a thread value, or an API call value for each node of the plurality of nodes.

3 . The method of claim 1 , wherein to update the at least one allocation of the at least one data group comprises:
identifying the at least one node of the plurality of nodes based at least on the resource metric of the at least one node indicating a relative level of capacity compared to at least one other node of the plurality of nodes.

4 . The method of claim 3 , wherein the relative level of capacity comprises a measure of utilization, availability, or load corresponding to the at least one resource indicator.

5 . The method of claim 1 , wherein the at least one data group is reallocated from an overloaded node of the plurality of nodes to a compatible node of the plurality of nodes, the compatible node identified based at least on a second resource metric of the compatible node, the second resource metric indicating a higher available resource capacity for the at least one data group compared to the plurality of nodes.

6 . The method of claim 1 , wherein to update the at least one allocation of the at least one data group of the plurality of data groups comprises:
updating, using the correction scheme corresponding to the second phase for a plurality of iterations, a plurality of allocations of the plurality of data groups among the plurality of nodes until the convergence parameter is satisfied.

7 . The method of claim 6 , wherein the convergence parameter corresponds to at least one of (i) an iteration limit corresponding to reallocation, (ii) an iteration parameter corresponding to a decrease in a sum of a plurality of resource metrics exceeding a resource parameter, (iii) at least one termination condition.

8 . The method of claim 1 , further comprising:
determining the resource metric of the at least one node based at least on applying a first scoring function to a plurality of first resource indicators of the at least one node; and determining a second resource metric of at least one second node based at least on applying a second scoring function to a plurality of second resource indicators of the at least one second node; wherein the first scoring function comprises a first weight prioritization of the plurality of first resource indicators based at least one first hardware configuration of the at least one node, and wherein the second scoring function comprises a second weight prioritization of the plurality of second resource indicators based at least one second hardware configuration of the at least one second node.

9 . The method of claim 8 , further comprising:
detect, by accessing an operating system (OS) or hardware interface of the at least one node and performing the at least one node command, the at least one first hardware configuration based at least on telemetry data of the at least one node returned from the performance of the at least one node command.

10 . The method of claim 1 , wherein each of the plurality of nodes operate independently from other nodes of the plurality of nodes, and wherein a plurality of first data groups allocated to a first node of the plurality of nodes is disjoint from a plurality of second data groups allocated to a second node of the plurality of nodes.

11 . The method of claim 1 , wherein to allocate the plurality of data groups to the plurality of nodes comprises:
assigning a plurality of data objects into the plurality of data groups, wherein the plurality of data groups satisfies a size parameter.

12 . The method of claim 11 , wherein to allocate the plurality of data groups to the plurality of nodes comprises:
aggregating a plurality of weights of the plurality of data objects of at least one of the plurality of data groups to generate a plurality of aggregated weights, each aggregated weight corresponding to a corresponding data group of the plurality of data groups; and generating an ordered structure comprising a plurality of virtual nodes, at least one virtual node corresponding to at least one of the plurality of nodes, the ordered structure used to allocate the plurality of data groups to the plurality of nodes based at least on the plurality of aggregated weights.

13 . The method of claim 1 , wherein the correction scheme corresponds to a plurality of reallocation operations to reallocate at least one of the plurality of data groups among the plurality of nodes based at least on a plurality of resource metrics.

14 . The method of claim 1 , wherein the allocation scheme corresponds to a plurality of allocation operations to (i) generate the plurality of data groups using a plurality of data objects and (ii) allocate the plurality of data groups based at least on at least one mapping function.

15 . A system, comprising:
at least one processor to execute operations comprising:
apply an allocation scheme to a plurality of data groups to cause an allocation of the plurality of data groups to a plurality of nodes; and
apply a correction scheme to the plurality of nodes to cause an update in at least one allocation of at least one data group of the plurality of data groups based at least on a convergence parameter and a resource metric, the resource metric corresponding to at least one resource indicator of a hardware configuration of at least one node of the plurality of nodes.

16 . The system of claim 15 , wherein the operations, when executed by the at least one processor, further cause the at least one processor to:
determine a resource indicator for each of the plurality of nodes based at least on the hardware configuration corresponding to at least one of a memory usage, a processor usage, a network usage, an application value, a thread value, or an API call value for each node of the plurality of nodes.

17 . The system of claim 15 , wherein the operations, when executed by the at least one processor, further cause the at least one processor to:
identify the at least one node of the plurality of nodes based at least on the resource metric of the at least one node indicating a relative level of capacity compared to at least one other node of the plurality of nodes.

18 . The system of claim 17 , wherein the relative level of capacity comprises a measure of utilization, availability, or load corresponding to the at least one resource indicator.

19 . The system of claim 15 , wherein the at least one data group is reallocated from an overloaded node of the plurality of nodes to a compatible node of the plurality of nodes, the compatible node identified based

Assignees

• Palantir Technologies Inc

Inventors

• Kumar Priyanshu
• Kong Jeremy

Classifications

• G06F9/5061

  Partitioning or combining of resources · CPC title

• G06F9/5016

  the resource being the memory · CPC title

• G06F9/505

  considering the load · CPC title

• G06F9/5072

  Grid computing · CPC title

• G06F9/5088

  involving task migration · CPC title