Data volume sculptor for deep learning acceleration

Patent metadata

Field	Value
Publication number	US-11977971-B2
Application number	US-202318167366-A
Country	US
Kind code	B2
Filing date	Feb 10, 2023
Priority date	Feb 27, 2018
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

A device include on-board memory, an applications processor, a digital signal processor (DSP) cluster, a configurable accelerator framework (CAF), and a communication bus architecture. The communication bus communicatively couples the applications processor, the DSP cluster, and the CAF to the on-board memory. The CAF includes a reconfigurable stream switch and data volume sculpting circuitry, which has an input and an output coupled to the reconfigurable stream switch. The data volume sculpting circuitry receives a series of frames, each frame formed as a two dimensional (2D) data structure, and determines a first dimension and a second dimension of each frame of the series of frames. Based on the first and second dimensions, the data volume sculpting circuitry determines for each frame a position and a size of a region-of-interest to be extracted from the respective frame, and extracts from each frame, data in the frame that is within the region-of-interest.

First claim

The invention claimed is:

1. A method, comprising:
receiving information at an input stream interface of a data volume sculpting circuit, the information including a series of frames, each frame formed as a two dimensional (2D) data structure;
determining a first dimension and a second dimension of each frame of the series of frames;
based on the first and second dimensions, determining for each frame a position and a size of a region-of-interest to be extracted from the respective frame; and
extracting from each frame, data in the frame that is within the region-of-interest.

2. The method of claim 1 , wherein the extracting includes:
for each datum in each frame that is outside the respective region-of-interest to be extracted from the respective frame, passing a null datum through an output interface of the data volume sculpting circuit.

3. The method according to claim 1 , wherein at least some frames of the series of frames have 2D data structures that include non-image feature data structures.

4. The method according to claim 1 , wherein the series of frames are received as a raw data stream having a start tag and a stop tag.

5. The method according to claim 1 , comprising:
analyzing a pair of two-dimensional coordinates to determine the position and the size of the region-of-interest to be extracted from the respective frame.

6. The method according to claim 1 , wherein determining the position and the size of the region-of-interest to be extracted from the respective frame includes analyzing a single point and a radius about the single point.

7. The method according to claim 1 , wherein determining the position and the size of the region-of-interest to be extracted from the respective frame includes analyzing a plurality of points and a distance between at least two of the plurality of points.

8. An integrated circuit, comprising:
on-board memory;
an applications processor;
a configurable accelerator framework (CAF); and
at least one communication bus architecture communicatively coupling the applications processor, and the CAF to the on-board memory, wherein the CAF includes:
a reconfigurable stream switch; and
data volume sculpting circuitry having an input interface coupled to the reconfigurable stream switch and an output interface coupled to the reconfigurable stream switch, wherein the data volume sculpting circuitry, in operation:

receives information at the input interface, the information including a series of frames, each frame formed as a two dimensional (2D) data structure;
determines a first dimension and a second dimension of each frame of the series of frames;
based on the first and second dimensions, determines for each frame a position and a size of a region-of-interest to be extracted from the respective frame; and
extracts from each frame, data in the frame that is within the region-of-interest.

9. The integrated circuit of claim 8 , wherein the data volume sculpting circuitry, in operation, for each datum in each frame that is outside the respective region-of-interest to be extracted from the respective frame, passes a null datum through the output interface of the data volume sculpting circuitry.

10. The integrated circuit according to claim 8 , wherein the data volume sculpting circuitry, in operation:
extracts the position and the size of the region-of-interest from each frame using the information received at the input stream interface.

11. The integrated circuit according to claim 8 , wherein the data volume sculpting circuitry comprises:
calculation circuitry, which, in operation, performs mathematical operations associated with determining the position and size of the respective regions of interest.

12. The integrated circuit according to claim 8 , wherein the data volume sculpting circuitry comprises:
a state machine, which, in operation, determines boundaries of the respective regions of interest.

13. A configurable accelerator framework (CAF), comprising:
a reconfigurable stream switch; and
data volume sculpting circuitry having an input interface coupled to the reconfigurable stream switch and an output interface coupled to the reconfigurable stream switch, wherein the data volume sculpting circuitry, in operation:
receives information at the input interface, the information including a series of frames, each frame formed as a two dimensional (2D) data structure;
determines a first dimension and a second dimension of each frame of the series of frames;
based on the first and second dimensions, determines for each frame a position and a size of a region-of-interest to be extracted from the respective frame; and
extracts from each frame, data in the frame that is within the region-of-interest.

14. The configurable accelerator framework of claim 13 , wherein the data volume sculpting circuitry, in operation, for each datum in each frame that is outside the respective region-of-interest to be extracted from the respective frame, passes a null datum through the output interface of the data volume sculpting circuitry.

15. The configurable accelerator framework of claim 13 , wherein the data volume sculpting circuitry, in operation:
extracts the position and the size of the region-of-interest from each frame using the information received at the input stream interface.

16. A non-transitory computer-readable medium having contents which configure configurable accelerator framework (CAF) circuitry to perform a method, the method comprising:
receiving information at an input stream interface of the CAF circuitry, the information including a series of frames, each frame formed as a two dimensional (2D) data structure;
determining a first dimension and a second dimension of each frame of the series of frames;
based on the first and second dimensions, determining for each frame a position and a size of a region-of-interest to be extracted from the respective frame; and
extracting from each frame, data in the frame that is within the region-of-interest.

17. The non-transitory computer-readable medium of claim 16 , wherein the extracting includes:
for each datum in each frame that is outside the respective region-of-interest to be extracted from the respective frame, passing a null datum through an output interface of the CAF circuitry.

18. The non-transitory computer-readable medium of claim 16 , wherein the series of frames are received as a raw data stream having a start tag and a stop tag.

19. The non-transitory computer-readable medium of claim 16 , wherein the method comprises:
extracting the position and the size of the region-of-interest from each frame using the information received at the input stream interface.

20. The non-transitory computer-readable medium of claim 16 , wherein the contents comprise instructions executed by the CAF circuitry.

Assignees

• St Microelectronics Srl
• St Microelectronics Int Nv

Inventors

• Singh Surinder Pal
• Boesch Thomas
• Desoli Giuseppe

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/063 Primary

  using electronic means · CPC title

• G06F9/3877

  using a secondary processor, e.g. coprocessor (peripheral processor G06F13/12) · CPC title

• G06F16/9024

  Graphs; Linked lists (G06F16/9027 takes precedence) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title