System and method for computing message digests

Patent metadata

Field	Value
Publication number	US-9619167-B2
Application number	US-201314091598-A
Country	US
Kind code	B2
Filing date	Nov 27, 2013
Priority date	Nov 27, 2013
Publication date	Apr 11, 2017
Grant date	Apr 11, 2017

Abstract

A data de-duplication approach leverages acceleration hardware in SSDs for performing digest computations used in de-duplication operations and support on behalf of an attached host, thereby relieving the host from the computing burden of the digest computation in de-duplication (de-dupe) processing. De-dupe processing typically involve computation and comparison of message digests (MD) and/or hash functions. Such MD functions are often also employed for cryptographic operations such as encryption and authentication. Often, SSDs include onboard hardware accelerators for MD functions associated with security features of the SSDs. However, the hardware accelerators may also be invoked for computing a message digest result and returning the result to the host, effectively offloading the burden of MD computation from the host, similar to an external hardware accelerator, but without redirecting the data since the digest computation is performed on a data stream passing through the SSD for storage.

First claim

What is claimed is:

1. A method of data duplication detection, comprising:
receiving a write request for a block of data at a storage device coupled to a host for providing non-volatile data storage;
receiving, from the host, a command to compute a message digest of the block of data in conjunction with fulfilling the write request;
invoking an onboard hardware accelerator configured for message digest computation based on the command from the host and applied to data in the block of data and traversing a data path from the host to a storage medium in a storage device, wherein the onboard hardware accelerator is an encryption engine disposed on the storage device and configured for encryption, decryption, and secure hash computation;
duplicating the block of data by snooping the data from a write stream of data to be written to the storage device;
computing, in the onboard hardware accelerator, a message digest of the data, by identifying the block of snooped data and applying a digest function to the block, from the traversed data path as the data passes on the data path from the host to the storage medium in the storage device for storage further comprising noninvasive snooping of the data to be stored, snooping occurring as the data is received at the storage device from the host, wherein the message digest computation is a stateless, flow-through computation applied to data as it passes from the post to the storage device for storage on a media component of the storage device;
storing the data on the storage medium in the storage device; and
returning the computed message digest from the onboard hardware accelerator to the host.

2. The method of claim 1 further comprising computing the message digest without redirecting the data from the data path to the storage medium.

3. The method of claim 1 further comprising identifying blocks of data previously written to the storage medium on the storage device;
retrieving the identified blocks into a digest buffer for offline computations;
invoking the hardware accelerators for computing a digest of the buffered data block; and
returning the computed digest to the host.

4. The method of claim 1 further comprising snooping the data block after receipt by the storage device from the host.

5. The method of claim 1 wherein the hardware accelerator snoops the data from the write stream as it is being written to the storage medium.

6. The method of claim 1 wherein the hardware accelerator snoops the data as the storage medium is invoked for mass storage.

7. A storage device comprising:
an interface from the storage device to a host device;
an onboard hardware accelerator configured for message digest computation by identification of a block of data and application of a digest function to the block based on a command from the host device and applied to data traversing a data path from the host device to a storage medium on the storage device, wherein the hardware accelerator is an encryption engine disposed on the storage device and configured for encryption, decryption, and secure hash computation;
a solid state storage medium to store the data;
the data path responsive to the hardware accelerators for computing a message digest of the data from the traversed data path as the data passes on the data path from the host to the storage medium for storage; and
the interface further configured to provide the computed message digest to the host device, the storage device being a solid state storage device configured to:
receive a write request for a block of data at the solid state storage device coupled to the host device for providing non-volatile data storage;
receive, from the host device, a command to compute a message digest of the block of data in conjunction with fulfilling the write request; and
duplicate the block of data by snooping the data from a write stream of data on the data path to be written to the solid state storage medium, further comprising noninvasive snooping of the data to be stored, snooping occurring as the data is received at the storage device from the host, wherein the message digest computation is a stateless, flow-through computation applied to data as it passes from the host to the storage device for storage on a media component of the storage device.

8. The storage device of claim 7 wherein the hardware accelerator is configured for computing the message digest without redirecting the data from the data path to the storage medium.

9. The storage device of claim 8 wherein the hardware accelerator is configured for noninvasive snooping of the data to be stored, snooping occurring as the data is received at the storage device from the host.

10. The storage device of claim 7 wherein the digest computation is a stateless, flow-through computation applied to data is it passes from the host to the storage device for storage on a media component of the storage device.

11. The storage device of claim 7 wherein the hardware accelerator further comprises inline and offline modules, the offline modules configured for:
identifying blocks of data previously written to the storage medium on the storage device;
retrieving the identified blocks into a digest buffer for offline computations;
invoking the hardware accelerators for computing a digest of the buffered data block; and
returning the computed digest to the host.

12. A method of storing data comprising:
augmenting a storage device with hardware accelerators for performing compression and security functions on data sent from a host to be stored on the storage device, wherein the hardware accelerators are encryption engines disposed on the storage device and configured for encryption, decryption, and secure hash computation;
receiving a write request for a block of data at a storage device coupled to the host for providing non-volatile data storage;
receiving, from the host, a command to compute a message digest of the block of data in conjunction with fulfilling the write request;
invoking the hardware accelerators based on a command from the host to compute a result by identifying the block of data and applying a digest function to the block as it passes on a data path from the host to the storage device for storage, including duplicating the block of data by snooping the data from a write stream of data on the data path to be written to the storage device, further comprising noninvasive snooping of the data to be stored, snooping occurring as the data is received at the storage device from the host, wherein the message digest computation is a stateless, flow-through computation applied to data is it passes from the host to the storage device for storage on a media component of the storage device; and
providing the computed result from the hardware accelerators to the host, the computation being a stateless, flow-through computation applied to data as it passes from the host to the storage device for storage on the storage device.

13. The method of claim 12 wherein computed result comprises a message digest.

14. A computer system comprising:
a solid state storage device (SSD) having a storage medium for storing data;
an onboard hardware accelerator on the SSD configured for message digest computation based on data traversing a data path to a storage medium on the storage device and on a command from a host, the data path responsive to the hardware accelerator to compute a message digest of the data from the traversed data path as the data passes on the data path to the storage medium for storage, wherein the hardware accelerator is an encryption engine disposed on the SSD and configured for

Assignees

• Intel Corp

Inventors

• Khan Jawad B
• Grimsrud Knut S
• Coulson Richard L

Classifications

• G06F3/0658

  Controller construction arrangements · CPC title

• G06F16/152

  using file content signatures, e.g. hash values · CPC title

• G06F3/0626

  Reducing size or complexity of storage systems · CPC title

• G06F3/061

  Improving I/O performance · CPC title

• G06F3/0659

  Command handling arrangements, e.g. command buffers, queues, command scheduling · CPC title