Automated syncing of data between security domains

Patent metadata

Field	Value
Publication number	US-11949653-B2
Application number	US-202218071002-A
Country	US
Kind code	B2
Filing date	Nov 29, 2022
Priority date	Jan 7, 2020
Publication date	Apr 2, 2024
Grant date	Apr 2, 2024

Abstract

Described herein are systems, methods, and non-transitory computer readable media for automating the transfer/syncing of datasets or other artifacts from one security domain (e.g., a low security side environment) to another security domain (e.g., a high security side environment) in a seamless manner that complies with requirements of a data transfer mechanism used to transfer data between the two security domains while ensuring data integrity and consistency between the two security domains.

First claim

What is claimed is:

1. A system in a first domain, comprising:
at least one processor; and
at least one memory storing computer-executable instructions, wherein the at least one processor is configured to access the at least one memory and execute the computer-executable instructions to:
receive, from a computing device in a second domain, a dataset and a command log associated with the dataset, wherein:
a security level of the first domain is higher than the security level of the second domain,
the dataset comprises a plurality of file segments generated from a source file in the second domain through a file preprocessing, and
the command log comprises commands executed in the file preprocessing;

generate data ingests by reversing the file preprocessing on the dataset according to the commands in the command log; and
trigger the data ingests to sync the target file in the first domain with the source file in the second domain.

2. The system of claim 1 , wherein the file preprocessing comprises:
partitioning the source file in the dataset into the plurality of file segments, each of the plurality of file segments being smaller than a threshold file size permitted for transferring data between the first domain and the second domain.

3. The system of claim 1 , wherein the file preprocessing comprises:
converting a file type of the source file into a different file type supported by the first domain.

4. The system of claim 1 , wherein the file preprocessing comprises:
performing data compression on the source file.

5. The system of claim 1 , wherein the command log further comprises:
a file path of the target file in the first domain, and
a file path of the source file in the second domain.

6. The system of claim 5 , wherein the file path of the target file comprises a relative file path of the target file that is different from an absolute file path of the target file in the first domain.

7. The system of claim 1 , wherein the reversing the file preprocessing on the dataset according to the commands in the command log comprises:
combining the plurality of file segments in the dataset according to the commands in the command log.

8. The system of claim 1 , wherein to sync the target file in the first domain with the source file in the second domain, the at least one processor is further configured to execute the computer-executable instructions to:
map the a file path of the target file in the first domain with a file path of the source file in the second domain.

9. The system of claim 1 , wherein to sync the target file in the first domain with the source file in the second domain, the at least one processor is further configured to execute the computer-executable instructions to:
create one or more copies of the target file for syncing to avoid overwriting the target file.

10. A method comprising:
receiving, from a computing device in a second domain, a dataset and a command log associated with the dataset, wherein:
a security level of the first domain is higher than the security level of the second domain,
the dataset comprises a plurality of file segments generated from a source file in the second domain through a file preprocessing, and
the command log comprises commands executed in the file preprocessing;

generating data ingests by reversing the file preprocessing on the dataset according to the commands in the command log; and
triggering the data ingests to sync the target file in the first domain with the source file in the second domain.

11. The method of claim 10 , wherein the file preprocessing comprises:
partitioning the source file in the dataset into the plurality of file segments, each of the plurality of file segments being smaller than a threshold file size permitted for transferring data between the first domain and the second domain.

12. The method of claim 10 , wherein the file preprocessing comprises:
converting a file type of the source file into a different file type supported by the first domain.

13. The method of claim 10 , wherein the file preprocessing comprises:
performing data compression on the source file.

14. The method of claim 10 , wherein the command log further comprises:
a file path of the target file in the first domain, and
a file path of the source file in the second domain.

15. The method of claim 14 , wherein the file path of the target file comprises a relative file path of the target file that is different from an absolute file path of the target file in the first domain.

16. The method of claim 10 , wherein the reversing the file preprocessing on the dataset according to the commands in the command log comprises:
combining the plurality of file segments in the dataset according to the commands in the command log.

17. The method of claim 10 , wherein the syncing of the target file in the first domain with the source file in the second domain comprises:
mapping the a file path of the target file in the first domain with a file path of the source file in the second domain.

18. The method of claim 10 , wherein the syncing of the target file in the first domain with the source file in the second domain comprises:
creating one or more copies of the target file for syncing to avoid overwriting the target file.

19. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:
receiving, from a computing device in a second domain, a dataset and a command log associated with the dataset, wherein:
a security level of the first domain is higher than the security level of the second domain,
the dataset comprises a plurality of file segments generated from a source file in the second domain through a file preprocessing, and
the command log comprises commands executed in the file preprocessing;

generating data ingests by reversing the file preprocessing on the dataset according to the commands in the command log; and
triggering the data ingests to sync the target file in the first domain with the source file in the second domain.

20. The non-transitory computer readable medium of claim 19 , wherein the file preprocessing comprises:
converting a file type of the source file into a different file type supported by the first domain.

Assignees

• Palantir Technologies Inc

Inventors

• Ochs-Willard Jess
• Brady Matthew
• Belardo Armando
• Skiles Mitchell

Classifications

• H04L63/0209 Primary

  Architectural arrangements, e.g. perimeter networks or demilitarized zones · CPC title

• G06F16/119

  Details of migration of file systems (migration mechanisms in storage systems G06F3/0647) · CPC title

• G06F16/164

  File meta data generation · CPC title

• G06F16/178

  Techniques for file synchronisation in file systems · CPC title

• H04L63/0428

  wherein the data content is protected, e.g. by encrypting or encapsulating the payload · CPC title