Methods, devices, and computer program products for processing data

Patent metadata

Field	Value
Publication number	US-11029866-B2
Application number	US-201916415812-A
Country	US
Kind code	B2
Filing date	May 17, 2019
Priority date	Oct 31, 2018
Publication date	Jun 8, 2021
Grant date	Jun 8, 2021

Abstract

A method comprises: determining whether an event triggering processing of data at a storage device occurs, the data being predetermined to be processed at a computing device associated with the storage device; in response to determining that the event occurs, determining available resources of the storage device; and in response to an amount of the available resources exceeding a first predetermined threshold, causing the storage device to process the data and provide the processed data to the computing device. As such, operations to be performed to data may be adaptively allocated, deployed and adjusted on the storage device and the computing device, so that computing capabilities of both the storage device and the computing device may be fully leveraged, and further the computing efficiency of the entire data processing system may be improved.

First claim

What is claimed is:

1. A method of processing data, comprising:
determining, by a storage device, whether an event triggering processing of data at the storage device occurs, the data being predetermined to be processed at a computing device associated with the storage device;
in response to determining that the event occurs, determining, by the storage device, available resources of the storage device;
in response to an amount of the available resources of the storage device exceeding a first predetermined threshold, causing the storage device to process the data rather than the computing device processing the data and provide the processed data from the storage device to the computing device;
determining, by the storage device, updated available resources of the storage device, the available resources being updated due to processing the data; and
in response to an amount of the updated available resources being below a second predetermined threshold, performing, by the storage device, at least one of:
pausing processing of the data; and
reducing operations of processing the data.

2. The method according to claim 1 , wherein determining whether the event triggering processing of data at the storage device occurs comprises at least one of:
in response to determining that the data is detected to be to-be-processed data stored in the storage device, determining that the event occurs; and
in response to determining that a request for processing the data is received from the computing device, determining that the event occurs.

3. The method according to claim 1 , wherein determining the available resources of the storage device comprises at least one of:
determining available computing resources of the storage device; and
determining available storage space of the storage device.

4. The method according to claim 1 , wherein processing the data comprises:
performing, on the data, at least one of: shuffling, decoding, re-sizing, and data augmentation.

5. The method according to claim 1 , further comprising:
notifying the computing device that the storage device processes the data.

6. The method according to claim 1 , wherein reducing the operation of processing the data comprises:
notifying the computing device of the reduced operations.

7. The method according to claim 1 , wherein processing the data comprises:
determining a negotiation result between the storage device and the computing device regarding processing the data, the negotiation result indicating which operations among operations of processing the data will be performed by the storage device; and
processing the data based on the negotiation result.

8. A computer program product being tangibly stored on a non-transient computer readable medium and comprising machine executable instructions which, when being executed, causing a machine to perform steps of the method of claim 1 .

9. A device for processing data, comprising at least one processing unit; at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when being executed by the at least one processing unit, causing the device to perform steps of the method of claim 1 .

10. A method of processing data, the method comprising:
sending, from a computing device to a storage device associated with the computing device, a request for processing data at the storage device, the data being predetermined to be processed at the computing device;
receiving, at the computing device, the processed data from the storage device, the data being processed by the storage device rather than the computing device processing the data when available resources of the storage device exceed a first predetermined threshold;
responsive to an amount of updated available resources of the storage device being below a second predetermined threshold, receiving, at the computing device, notification of a reduction in operations of processing the data; and
performing, by the computing device, at least a portion of the operations of processing the data.

11. The method according to claim 10 , wherein sending the request comprises:
determining available resources of the computing device; and
in response to an amount of the available resources of the computing device being below a third predetermined threshold, sending the request to the storage device.

12. The method according to claim 11 , wherein determining the available resources of the computing device comprises at least one of:
determining available computing resources of the computing device; and
determining available storage space of the computing device.

13. The method according to claim 10 , further comprising:
determining a negotiation result between the computing device and the storage device regarding processing the data, the negotiation result indicating which operations among operations of processing the data will be performed by the storage device.

14. The method according to claim 10 , wherein processing the data comprises:
performing, on the data, at least one of: shuffling, decoding, re-sizing, and data augmentation.

15. A computer program product being tangibly stored on a non-transient computer readable medium and comprising machine executable instructions which, when being executed, causing a machine to perform steps of the method of claim 10 .

16. A device for processing data, comprising:
at least one processing unit;
at least one memory coupled to the at least one processing unit and storing instructions to be executed by the at least one processing unit, the instructions, when being executed by the at least one processing unit, causing the device to perform acts comprising:
sending, from a computing device to a storage device associated with the computing device, a request for processing data at the storage device, the data being predetermined to be processed at the computing device;
receiving, at the computing device, the processed data from the storage device, the data being processed by the storage device rather than the computing device processing the data when available resources of the storage device exceed a first predetermined threshold;
responsive to an amount of updated available resources of the storage device being below a second predetermined threshold, receiving, at the computing device, notification of a reduction in operations of processing the data; and
performing, by the computing device, at least a portion of the operations of processing the data.

17. The device according to claim 16 , wherein sending the request comprises:
determining available resources of the computing device; and
in response to an amount of the available resources of the computing device being below a third predetermined threshold, sending the request to the storage device.

18. The device according to claim 17 , wherein determining the available resources of the computing device comprises at least one of:
determining available computing resources of the computing device; and
determining available storage space of the computing device.

19. The device according to claim 16 , the acts further comprising:
determining a negotiation result between the computing device and the storage device regarding processing the data, the negotiation result indicating which operations among operations of processing the data will be performed by the storage device.

Assignees

• Emc Ip Holding Co Llc

Inventors

• Zhao Junping
• Hou Andrew Anzhou

Classifications

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• G06F3/0653

  Monitoring storage devices or systems · CPC title

• G06N20/00

  Machine learning · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title

• G06F3/067

  Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS] · CPC title