Patch memory system

Patent metadata

Field	Value
Publication number	US-9934153-B2
Application number	US-201514788593-A
Country	US
Kind code	B2
Filing date	Jun 30, 2015
Priority date	Jun 30, 2015
Publication date	Apr 3, 2018
Grant date	Apr 3, 2018

Abstract

A patch memory system for accessing patches from a memory is disclosed. A patch is an abstraction that refers to a contiguous, array of data that is a subset of an N-dimensional array of data. The patch memory system includes a tile cache, and is configured to fetch data associated with a patch by determining one or more tiles associated with an N-dimensional array of data corresponding to the patch, and loading data for the one or more tiles from the memory into the tile cache. The N-dimensional array of data may be a two-dimensional (2D) digital image comprising a plurality of pixels. A patch of the 2D digital image may refer to a 2D subset of the image.

First claim

What is claimed is:

1. A system, comprising:
one or more processors; and
a patch memory system coupled to the one or more processors, the patch memory system comprising:
a memory interface configured to receive instructions for accessing patches in a memory;
a patch unit that includes a patch table for storing data corresponding to one or more patches;
an address translation unit for generating physical memory addresses for locations in the memory allocated to data associated with the patches;
an image table for storing data structures corresponding to one or more images that are associated with one or more tiles; and
a tile cache,

wherein the patch memory system is configured to fetch data associated with a patch by:
identifying the one or more tiles associated with an N-dimensional array of data corresponding to the patch, and
loading data for the one or more tiles from the memory into the tile cache.

2. The system of claim 1 , wherein the N-dimensional array is an image comprising a plurality of pixels in a two-dimensional array.

3. The system of claim 2 , wherein the memory interface includes:
a vector/word access unit for processing a first set of instructions to access data words in the memory;
a tile access unit for processing a second set of instructions to access tiles in the memory; and
a patch access unit for processing a third set of instructions to access patches in the memory.

4. The system of claim 1 , wherein the patch unit further includes a patch shift engine configured to process instructions that cause an origin for a particular patch defined in the patch table to be shifted to a new location relative to an origin for a corresponding image in the one or more images.

5. The system of claim 4 , wherein the patch shift engine implements at least one of a sliding window trace pattern, a sliding window Z pattern, and a sliding window diagonal pattern.

6. The system of claim 1 , wherein the address translation unit includes:
a two-dimensional translation unit configured to generate memory addresses corresponding to two-element vectors;
a three-dimensional translation unit configured to generate memory addresses corresponding to three-element vectors; and
a boundary handling unit configured to generate memory addresses for pixels inside the boundary of an image based on coordinates located outside the boundary of the image.

7. The system of claim 1 , wherein the tile cache includes a plurality of memory banks, and wherein image data associated with a tile is arranged in the plurality of memory banks such that pixel data for a plurality of adjacent pixels in a row or pixel data for a plurality of adjacent pixels in a column can be accessed in a single clock cycle.

8. The system of claim 1 , wherein the tile cache includes a patch-to-tile unit configured to identify one or more tiles in the image associated with a particular patch based on an origin of the patch relative to an origin of an image and a size of the particular patch.

9. The system of claim 1 , wherein a patch is specified by one or more fields included in the patch table, the one or more fields including at least an Origin field, a Height field, and a Width field.

10. The system of claim 9 , wherein the Origin field comprises a two-element vector that specifies an x coordinate value and a y coordinate value relative to an origin of a source image associated with the patch.

11. The system of claim 1 , wherein the one or more processors comprise a plurality of vector processing units, each vector processing unit including a plurality of lanes for processing data in parallel.

12. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause the processor to perform steps comprising:
receiving, by a memory interface, an instruction for accessing data associated with a patch in a memory;
identifying, by a patch unit that includes a patch table for storing data corresponding to one or more patches, the patch;
identifying, by a tile access unit, one or more tiles associated with an N-dimensional array of data corresponding to the patch;
generating, by an address translation unit, a physical memory address for a location in the memory allocated to data associated with the patch;
accessing an image table that stores a data structure corresponding to one or more images that are associated with the one or more tiles; and
loading data for the one or more tiles from the location in the memory into a tile cache included in the processor.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the N-dimensional array is an image comprising a plurality of pixels in a two-dimensional array.

14. The non-transitory computer-readable storage medium of claim 12 , wherein a patch is specified by one or more fields included in the patch table, the one or more fields including at least an Origin field, a Height field, and a Width field.

15. The non-transitory computer-readable storage medium of claim 14 , wherein the Origin field comprises a two-element vector that specifies an x coordinate value and a y coordinate value relative to an origin of a source image associated with the patch.

16. A programmable image signal processor, comprising:
one or more vector processing units; and
a patch memory system coupled to the one or more vector processing units, the patch memory system comprising:
a memory interface configured to receive instructions for accessing patches in a memory;
a patch unit that includes a patch table for storing data corresponding to one or more patches;
an address translation unit for generating physical memory addresses for locations in the memory allocated to data associated with the patches;
an image table for storing data structures corresponding to one or more images that are associated with one or more tiles; and

a tile cache,
wherein the patch memory system is configured to fetch data associated with a patch by:
identifying the one or more tiles associated with an N-dimensional array of data corresponding to the patch, and
loading data for the one or more tiles from the memory into the tile cache.

17. The programmable image signal processor of claim 16 , wherein the N-dimensional array is an image comprising a plurality of pixels in a two-dimensional array.

18. The programmable image signal processor of claim 16 , wherein the memory interface includes:
a vector/word access unit for processing a first set of instructions to access data words in the memory;
a tile access unit for processing a second set of instructions to access tiles in the memory; and
a patch access unit for processing a third set of instructions to access patches in the memory.

Assignees

• Nvidia Corp

Inventors

• Clemons Jason Lavar
• Cheng Chih-Chi
• Johnson Daniel Robert
• Keckler Stephen William
• Frosio Iuri
• Tsai Yun-Ta

Classifications

• G06F12/0893 Primary

  Caches characterised by their organisation or structure · CPC title

• G06F2212/454

  Vector or matrix data · CPC title

• G06F2212/455

  Image or video data · CPC title