Interleaved processing of template data and search data according to a search window

Patent metadata

Field	Value
Publication number	US-12573171-B2
Application number	US-202418431565-A
Country	US
Kind code	B2
Filing date	Feb 2, 2024
Priority date	Feb 2, 2024
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

A processor, method, and non-transitory computer-readable storage medium for processing template data and search data according to a search window applied to the search data. The search window comprising a set of offset positions. The processing is performed by a block matching engine (BME) that produces a tensor with difference values, and a convolutional engine (CE) that performs a convolutional operation on the tensor. The processing is performed in an iterative interleaved fashion, by dividing the set of offset positions into a plurality of subsets of offset positions. In parallel with processing of the first X offset positions by the CE, the BME generates the next X channels of the tensor, and which are subsequently pipelined through to the CE via an internal storage, etc.

First claim

What is claimed is:

1 . A processor for processing template data and search data according to a search window applied to the search data, the search window comprising a set of offset positions, the processor comprising an internal memory, a handling unit, a convolutional engine, CE, and a block matching engine, BME, wherein the BME being configured to indicate a measure of similarity between template data and search data,
the BME configured for:
receiving first invocation data indicating a plurality of offset positions, and an output storage element; and
determining a tensor having a plurality of channels, by, for each offset position, offsetting the search data according to the offset position, determining difference values between the template data and the offset search data, writing the difference values in a channel of the tensor; and storing the tensor in the output storage element;
the CE configured for:
receiving second invocation data, the second invocation data identifying an input storage element, reading a tensor having a plurality of channels from the input storage element, and performing a convolutional operation on the tensor;
the handling unit configured for:
accessing a plurality of subsets of offset positions from the set of offset positions;
performing interleaved iterative processing of the plurality of subsets of offset positions, using the CE and the BME, comprising, for each iteration:
dispatching first invocation data to the BME identifying a subset of offset positions among the plurality of subsets of offset positions; and an output storage element in the internal memory; and
dispatching second invocation data to the CE identifying, as input storage element, the output storage element as identified in first invocation data dispatched to the BME in a previous iteration;

wherein the processor is configured for storing a result from the convolutional operation performed by the CE in internal memory.

2 . The processor of claim 1 , wherein the internal memory comprises a first and a second storage element, wherein the handling unit is configured for performing the interleaved iterative processing of the plurality of subsets of offset position by:
for a first subset of offsets positions, dispatching first invocation data to the BME, the invocation data defining the first subset of offsets positions and the first storage element as output storage element; for each subset of offsets positions excluding the first sub-set of offsets positions:
alternating between using the first and the second storage element as output storage element for the BME, and using the other one of the first and second storage element as the input storage element of the CE;
dispatching second invocation data to the CE identifying the input storage element according to the alternating;
dispatching first invocation data to the BME indicating the subset of offsets positions and the output storage element according to the alternating.

3 . The processor of claim 1 , wherein each subset of offset positions comprises a predetermined number of offset positions.

4 . The processor of claim 3 , wherein the predetermined number of offset positions is a multiple of 8.

5 . The processor of claim 3 , wherein one or more of the offset positions in the set of offset positions are skipped in the plurality of subset of offset positions, such that each subset comprises the predetermined number of offset positions.

6 . The processor of claim 5 , wherein the set of offset positions is ordered according to a predefined search pattern, wherein the skipping one or more of the offset positions comprises skipping one or more offset positions ordered last in the ordered set of offset positions.

7 . The processor of claim 1 , wherein each subset comprises continuous offset positions from the search window in one of a: vertical direction or horizontal direction.

8 . The processor of claim 1 , further comprises a motion vector determining unit configured to read the stored output from the CE, and process the output to determine the plurality of motion vectors defining data movements between the template data and search data.

9 . The processor of claim 1 , wherein search window is one of: rectangular, elliptical, or circular.

10 . The processor of claim 1 , wherein the BME is configured for, prior to storing the tensor in the output storage element, downscaling each difference value by applying a predefined scaling equation to the difference value.

11 . The processor of claim 10 , wherein the BME is configured for applying the predefined scaling equation to the difference value by computing the square root of the difference value.

12 . The processor of claim 1 , wherein the template data and the search data each comprises a 2D tensor.

13 . A method performed by a processor when processing template data and search data according to a search window applied to the search data, the search window comprising a set of offset positions, the processor comprising an internal memory; the method comprising:
accessing a plurality of subsets of offset positions from the set of offset positions; performing interleaved iterative processing of the plurality of subsets of offset positions, comprising, for each iteration:
dispatching first invocation data identifying a subset of offset positions among the plurality of subsets of offset positions; and an output storage element in the internal memory;
determining a tensor having a plurality of channels, by, for each offset position of the subset of offset positions identified in the first invocation data, offset the search data according to the offset position, determining difference values between the template data and the offset search data, writing the difference values in a channel of the tensor; and storing the tensor in the output storage element identified in the first invocation data;
dispatching second invocation data identifying, as an input storage element, the output storage element as identified in first invocation data dispatched in a previous iteration;
reading a tensor having a plurality of channels from the input storage element identified in the second invocation data, performing a convolutional operation on the tensor; and
storing a result from the convolutional operation in internal memory.

14 . The method of claim 13 , wherein the internal memory comprises a first and a second storage element, wherein performing the interleaved iterative processing of the plurality of subsets of offset position comprises:
for a first subset of offsets positions, dispatching first invocation data the invocation data defining the first subset of offsets positions and the first storage element as output storage element; for each subset of offsets positions excluding the first sub-set of offsets positions:
alternating between using the first and the second storage element as output storage element of first invocation data, and using the other one of the first and second storage element as the input storage element second invocation data;
dispatching second invocation identifying the input storage element according to the alternating;
dispatching first invocation data indicating the subset of offsets positions and the output storage element according to the alternating.

15 . The method of claim 13 , wherein each subset of offset positions comprises a predetermined number of offset positions.

16 . The method of claim 15 , wherein one or more of the offset positions in the set of offset positions are skipped in the plurality of offset positions,

Assignees

• Advanced Risc Mach Ltd

Inventors

• Brothers Iii John Wakefield
• Ünal Metin Gokhan
• VENU Balaji
• Holm Rune

Classifications

• G06V10/82 Primary

  using neural networks · CPC title

• G06V2201/07

  Target detection · CPC title

• G06V10/74

  Image or video pattern matching; Proximity measures in feature spaces · CPC title

• G06T7/223

  using block-matching · CPC title

• G06V10/469 Primary

  Contour-based spatial representations, e.g. vector-coding · CPC title