Selectively shearing data when manipulating data during record processing

Patent metadata

Field	Value
Publication number	US-11675513-B2
Application number	US-202117403617-A
Country	US
Kind code	B2
Filing date	Aug 16, 2021
Priority date	Aug 16, 2021
Publication date	Jun 13, 2023
Grant date	Jun 13, 2023

Abstract

A computer-implemented method, according to one embodiment, includes: storing records in an input data buffer, where each of the records include a key which is appended to payload data in the respective record. Moreover, for each of the records: shearing the key associated with the record from the payload data, normalizing the sheared key, and storing the normalized sheared key in a first target area of memory. A determination is made as to whether a size of the payload data in the record is outside a predetermine range, and in response to determining that the size of the payload data in the record is outside the predetermine range, the payload data is stored in a second target area of memory. A data locator is also appended to the normalized sheared key in the first target area of memory to form a sheared record.

First claim

What is claimed is:

1. A computer-implemented method for reducing data drag when manipulating data during record processing, the method comprising:
storing records in an input data buffer, wherein each respective record, of the records, includes a key which is appended to payload data in the respective record; and
processing the records in the input data buffer by performing the following operations on each of the respective records:
shearing the key associated with the respective record from the payload data in the respective record,
normalizing the sheared key,
storing the normalized sheared key in a first target area of memory,
in response to determining that the respective record is not a variable length record, determining whether a size of the payload data in the respective record is outside a predetermine range,
in response to determining that the size of the payload data in the respective record is outside the predetermine range, storing the payload data in a second target area of memory, and
appending a data locator to the normalized sheared key, in the first target area of memory, to form a sheared record, wherein the data locator corresponds to a storage location of the payload data in the second target area of memory.

2. The computer-implemented method of claim 1 , comprising:
processing the records in the input data buffer by performing the following operations on each of the respective records:
in response to determining that the size of the payload data in the respective record is not outside the predetermine range, combining the normalized sheared key with the payload data to form an appended record, and
storing the appended record in the first target area of memory.

3. The computer-implemented method of claim 2 , comprising:
causing a sorting operation to be performed on the appended records in the first target area of memory.

4. The computer-implemented method of claim 1 , comprising:
causing a sorting operation to be performed on the sheared records in the first target area of memory.

5. The computer-implemented method of claim 1 , wherein normalizing the sheared key includes:
attaching a stability index to the sheared key, wherein the stability index maintains a relative order of the payload data in the respective record with respect to other payload data in the second target area of memory.

6. The computer-implemented method of claim 1 , wherein normalizing the sheared key includes:
determining whether a type of information in the key, associated with the respective record, is supported;
in response to determining that the type of information in the key, associated with the respective record, is not supported, determining whether the information in the key, associated with the respective record, is properly aligned; and
in response to determining that the information in the key, associated with the respective record, is not properly aligned, normalizing the sheared key.

7. The computer-implemented method of claim 1 , comprising:
processing the records in the input data buffer by performing the following operations on each of the respective records:
determining whether the respective record is a variable length record; and
in response to determining that the respective record is a variable length record:

storing the payload data in the second target area of memory, and appending the data locator to the normalized sheared key in the first target area of memory to form the sheared record.

8. A computer program product for reducing data drag when manipulating data during record processing, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable and/or executable by a processor to cause the processor to:
store, by the processor, records in an input data buffer, wherein each respective record, of the records, includes a key which is appended to payload data in the respective record; and
process, by the processor, the records in the input data buffer by performing the following operations on each of the respective records:
shear, by the processor, the key associated with the respective record from the payload data in the respective record,
normalize, by the processor, the sheared key,
store, by the processor, the normalized sheared key in a first target area of memory,
in response to determining that the respective record is not a variable length record, determine, by the processor, whether a size of the payload data in the respective record is outside a predetermine range,
in response to determining that the size of the payload data in the respective record is outside the predetermine range, store, by the processor, the payload data in a second target area of memory, and
append, by the processor, a data locator to the normalized sheared key, in the first target area of memory, to form a sheared record, wherein the data locator corresponds to a storage location of the payload data in the second target area of memory.

9. The computer program product of claim 8 , wherein the program instructions are readable and/or executable by the processor to cause the processor to:
process, by the processor, the records in the input data buffer by performing the following operations on each of the respective records:
in response to determining that the size of the payload data in the respective record is not outside the predetermine range, combine, by the processor, the normalized sheared key with the payload data to form an appended record, and
store, by the processor, the appended record in the first target area of memory.

10. The computer program product of claim 9 , wherein the program instructions are readable and/or executable by the processor to cause the processor to:
cause, by the processor, a sorting operation to be performed on the appended records in the first target area of memory.

11. The computer program product of claim 8 , wherein the program instructions are readable and/or executable by the processor to cause the processor to:
cause, by the processor, a sorting operation to be performed on the sheared records in the first target area of memory.

12. The computer program product of claim 8 , wherein normalizing the sheared key includes:
attaching a stability index to the sheared key, wherein the stability index maintains a relative order of the payload data in the respective record with respect to other payload data in the second target area of memory.

13. The computer program product of claim 8 , wherein normalizing the sheared key includes:
determining whether a type of information in the key, associated with the respective record, is supported;
in response to determining that the type of information in the key, associated with the respective record, is not supported, determining whether the information in the key, associated with the respective record, is properly aligned; and
in response to determining that the information in the key, associated with the respective record, is not properly aligned, normalizing the sheared key.

14. The computer program product of claim 8 , wherein the program instructions are readable and/or executable by the processor to cause the processor to:
process, by the processor, the records in the input data buffer by performing the following operations on each of the respective records:
determine, by the processor, whether the respective record is a variable length record; and
in response to determining that the respective record is a variable length record:
store, by the processor, the payload data in the second target area o

Assignees

• IBM

Inventors

• Compton Scott B
• Suarez Jeffrey Richard
• Pardini Matthew Michael Garcia
• Jacobi Christian
• Steenken Dominik
• Kolusu Sri Hari
• Vezinaw Vicky

Classifications

• G06F3/067 Primary

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title

• G06F3/0685

  Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays · CPC title

• G06F3/0643 Primary

  Management of files · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/064

  Management of blocks · CPC title