Systems and methods for generating metadata describing unstructured data objects at the storage edge

Patent metadata

Field	Value
Publication number	US-11068544-B2
Application number	US-201916263387-A
Country	US
Kind code	B2
Filing date	Jan 31, 2019
Priority date	Jul 31, 2018
Publication date	Jul 20, 2021
Grant date	Jul 20, 2021

Abstract

A storage control device coupled to a storage device and located remotely from a host device receives media object data from the host device. The storage control device identifies a type of the media object data and select, based on the identified type, a computational model from among a plurality of computational models for use by a computational engine of the storage control device. The computational engine uses the selected computational model to generate metadata describing the media object data. The metadata is stored in the storage device so as to be selectively retrievable from the storage device separately from the media object data.

First claim

What is claimed is:

1. A method for storing and managing media object data, the method comprising:
receiving, at a storage control device, media object data, the storage control device being located at a storage edge remotely from a host device and coupled to a storage device;
identifying, at the storage control device, using control circuitry of the storage control device, a type of the media object data;
selecting, at the storage control device, based on the identified type, a computational model from among a plurality of computational models for use by a computational engine of the storage control device;
generating, by the computational engine, using the selected computational model, metadata describing the media object data;
providing the generated metadata to the storage device for storage of the metadata so as to be selectively retrievable from the storage device separately from the media object data.

2. The method of claim 1 , further comprising:
retrieving input data interface parameters associated with the selected computational model, the input data interface parameters identifying a data format for which the selected computational model is configured;
accessing the media object data at the storage edge using the input data interface parameters; and
converting the media object data to the identified data format.

3. The method of claim 1 , further comprising:
retrieving, by the storage control device, hardware interface parameters associated with the selected computational model;
applying the hardware interface parameters to a hardware interface;
accessing, at the storage edge, the media object data from a first physical connection using the hardware interface; and
storing the media object data and metadata in the storage device through a second physical connection using the hardware interface.

4. The method of claim 1 , wherein the media object data is encrypted, the method further comprising:
retrieving, by the storage control device, decryption parameters associated with the selected computational model; and
decrypting, at the storage edge, the media object data using the decryption parameters.

5. The method of claim 1 , wherein the media object data is retrieved from the storage device, the method further comprising;
identifying the computational model used to generate the metadata, the computational model being a first computational model and the metadata being first metadata;
selecting a second computational model that is different from the first computational model to generate second metadata that is different from the first metadata; and
storing the second metadata to be selectively retrievable from the first metadata and from the media object data.

6. The method of claim 5 , further comprising:
combining the first metadata and the second metadata to generate combined metadata; and
providing the combined metadata to the storage device for storage of the combined metadata so as to be selectively retrievable from the storage device separately from the media object data.

7. The method of claim 1 , further comprising:
receiving, at the storage edge, from the host device, a request for media object data containing objects meeting one or more criteria.

8. The method of claim 7 , further comprising, in response to the request for media object data containing objects meeting the one or more criteria:
retrieving, from the storage device, only the metadata describing the media object data separately from the media object data;
identifying, based on the retrieved metadata, one or more portions of the media object data that meet the one or more criteria;
retrieving, at the storage edge, only the identified one or more portions of the media object data from the storage device separately from other portions of the media object data; and
sending to the host device, over a computer network, retrieved portions of media object data containing objects meeting the one or more criteria.

9. The method of claim 1 , further comprising:
generating a copy of the media object data; and
concurrently providing, by the storage control device, the media object data to the storage device for storage and providing the copy of the media object data to the computational engine for processing to generate metadata.

10. The method of claim 1 , further comprising:
storing the media object data and the metadata in a logically separable manner so as to be selectively retrievable separately from one another.

11. A data storage controller, located at a storage edge remotely from a host device and coupled to one or more storage devices, the data storage controller comprising:
control circuitry local to the storage device configured to:
receive media object data for storage at the storage device; and
identify a type of the media object data; and

a computational engine associated with the control circuitry, the computational engine configured to:
select, based at least on the identified type, a computational model from among a plurality of computational models for use by the computational engine to generate metadata describing the media object data;
generate, using the selected computational model, metadata describing the media object data; and
provide the generated metadata to the storage device for storage of the metadata so as to be selectively retrievable from the storage device separately from the media object data.

12. The data storage controller of claim 11 , wherein the control circuitry is further configured to:
retrieve input data interface parameters associated with the selected computational model, the input data interface parameters identifying a format for which the selected computational model is configured;
access the media object data at the storage edge using the input data interface parameters; and
convert the media object data to the identified data format.

13. The data storage controller of claim 11 , wherein the control circuitry is further configured to:
retrieve hardware interface parameters associated with the selected computational model;
apply the hardware interface parameters to a hardware interface;
access the media object data at the storage edge through a first physical connection using the hardware interface; and
store the media object data and metadata in the storage device through a second physical connection using the hardware interface.

14. The data storage controller of claim 11 , wherein the control circuitry is further configured to:
retrieve decryption parameters associated with the selected computational model; and
decrypt, at the storage edge, the media object data using the decryption parameters.

15. The data storage controller of claim 11 , wherein the media object data is retrieved from the storage device, and wherein the control circuitry is further configured to:
identify the computational model used to generate the metadata, the computational model being a first computational model and the metadata being first metadata;
select a second computational model that is different from the first computational model to generate second metadata that is different from the first metadata; and
store the second metadata to be selectively retrievable from the first metadata and from the media object data.

16. The data storage controller of claim 15 , wherein the control circuitry is further configured to:
combine the first metadata and the second metadata to generate combined metadata; and
provide the combined metadata to the storage device for storage of the combined metadata so as to be selectively retrievable

Assignees

• Marvell Int Ltd
• Marvell Asia Pte Ltd

Inventors

• Kudryavtsev Konstantin
• Mizrahi Noam
• Oberg Mats
• Varnica Nedeljko
• Nangare Nitin
• Polivanyi Igor
• Ratnayake Ruwan
• Jiang Leo
• Chau Quynh
• Chang Wen Lung

Classifications

• G06N3/096

  Transfer learning · CPC title

• G06F16/907 Primary

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06F15/17331

  Distributed shared memory [DSM], e.g. remote direct memory access [RDMA] · CPC title

• G06F16/483

  using metadata automatically derived from the content · CPC title

• G06F3/0688

  Non-volatile semiconductor memory arrays · CPC title