Transformation in tabular data cleaning tool

Patent metadata

Field	Value
Publication number	US-11645453-B2
Application number	US-202217808239-A
Country	US
Kind code	B2
Filing date	Jun 22, 2022
Priority date	Jun 1, 2018
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

A system including first computer memory storing a full data set representable in rows and columns, a second computer memory storing executable instructions, and processors configured to execute the instructions to cause presentation of data of the full data set on a display including columns of data each having data fields, receive user input identifying a column of the data set, determine items to modify in information in the data fields of the identified column, generate and cause display of an indication of a proposed change action to modify the determined items, and in response to a user input indicating a selection of the indication of the proposed change action, update the presentation of the data based on the change action to modify information displayed in the data fields of the identified column of the data, and store a log of the change action.

First claim

What is claimed is:

1. A system, comprising:
one or more computer processors configured to execute computer-executable instructions to cause the system to at least:
generate and cause display, on a display device, data arranged in tabular format including a plurality of columns and rows representing a subset of data that is a portion of a first data set, the displayed columns showing only a portion of the first data set, the first data set including data field information;
in response to a selection of a portion of the displayed columns, generate and cause display of data metrics, of the first data set, for data field information in the selected portion of the columns for all the rows including rows of the selected columns that are not displayed;
generate and cause display of a change action to modify data field information of the first data set; and
in response to a selection of the change action, cause update of the data metrics on the display device, the update of the data metrics reflecting changes that would be made in the first data set caused by the selected change action without modifying the first data set.

2. The system of claim 1 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to store a log of the change action.

3. The system of claim 1 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to:
access the log;
apply each change action in the log to the first data set; and
save a second data set, the second data set including modifications made to the first data set based on the change action in the log.

4. The system of claim 2 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to iteratively:
receive user input indicating a selection of one of a plurality of displayed change actions; and
store a log of selected change actions.

5. The system of claim 4 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to:
access the log;
apply each change action in the log to the first data set; and
save a second data set, the second data set including modifications made to the first data set based on the change action in the log.

6. The system of claim 4 , wherein the one or more computer hardware processors are further configured to execute the computer-executable instructions to:
access the log;
apply each change action stored in the log to a second data set to create a modified second data set; and
save the modified second data set as a third data set, the third data set including all of the modifications made to the second data set by the change actions stored in the log.

7. The system of claim 1 , wherein the plurality of rows of the first data set includes more than tens thousand of rows of data.

8. The system of claim 1 , wherein the change action includes a suggested correction to information in data fields of the identified column and an indicator of importance of the suggested correction.

9. The system of claim 1 , wherein the change action includes modifying the data field information to at least one of: changing the spelling of a word; changing the case of letters; deleting a space; adding a space; deleting a period, comma, semi-colon, or colon; or adding a period, comma, semi-colon, or colon.

10. The system of claim 1 , wherein the change action comprises replacing data fields that include first information with second information.

11. The system of claim 1 , wherein the change action includes changing a data type of at least one data field in the first data set.

12. The system of claim 1 , wherein cause update of the data metrics on the display comprises concatenating at least one alphanumeric character or punctuation to information in a plurality of data fields.

13. A method of preparing tabular representable data for further processing, the method comprising:
generating and causing display on a display device data arranged in tabular format including a plurality of columns and rows representing a subset of data that is a portion of a first data set, the displayed columns showing only a portion of the first data set, the first data set including data field information;
in response to a selection of a portion of the displayed columns, generating and causing display of data metrics, of the first data set, for data field information in the selected portion of the columns for all the rows including rows of the selected columns that are not displayed;
generating and causing display of a change action to modify data field information of the first data set; and
in response to a selection of the change action, causing update of the data metrics on the display device, the update of the data metrics reflecting changes that would be made in the first data set caused by the selected change action without modifying the first data set.

14. The method of claim 13 , further comprising storing a log of the change action.

15. The method of claim 14 , further comprising:
accessing the log;
applying each change action in the log to the first data set; and
saving a second data set, the second data set including modifications made to the first data set based on the change action in the log.

16. The method of claim 14 , further comprising:
accessing the log;
applying each change action stored in the log to a second data set to create a modified second data set; and
saving the modified second data set as a third data set, the third data set including all of the modifications made to the second data set by the change actions stored in the log.

17. The method of claim 13 , wherein the plurality of rows of the first data set includes more than tens thousand of rows of data.

18. The method of claim 13 , wherein the change action includes a suggested correction to information in data fields of the identified column and an indicator of importance of the suggested correction.

19. The method of claim 13 , further comprising generating and causing display of tool buttons that when selected allow filtering operations on selected data.

20. The method of claim 13 , wherein the change action includes modifying the data field information to at least one of: changing the spelling of a word; changing the case of letters; deleting a space; adding a space; deleting a period, comma, semi-colon, or colon; or adding a period, comma, semi-colon, or colon.

Assignees

• Palantir Technologies Inc

Inventors

• Cervelli Daniel
• Storr Adam
• Rickard Jesse
• Memon A Rafee

Classifications

• G06F16/221

  Column-oriented storage; Management thereof · CPC title

• G06F40/18

  of spreadsheets (form-filling G06F40/174) · CPC title

• G06F16/2358

  Change logging, detection, and notification (replication G06F16/27) · CPC title

• G06F40/166 Primary

  Editing, e.g. inserting or deleting · CPC title

• G06F16/215

  Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors · CPC title