Circuit for performing normalized cross correlation

Patent metadata

Field	Value
Publication number	US-10997736-B2
Application number	US-201816100780-A
Country	US
Kind code	B2
Filing date	Aug 10, 2018
Priority date	Aug 10, 2018
Publication date	May 4, 2021
Grant date	May 4, 2021

Abstract

Embodiments relate to a normalized cross correlation (NCC) circuit that can perform a normalized cross correlation between input patch data and kernel data. An interface circuit of an image signal processor receives input patch data from a source. Input patch data is data that represents a portion of a frame of image data from the source. The NCC circuit includes a filtering circuit and a normalization circuit. The filtering circuit receives the input patch data from the interface circuit and performs a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data. The normalization circuit computes a normalized score output based on the convolution output data and the kernel data. The normalized score output includes normalization scores for each location of the convolution output data.

First claim

What is claimed is:

1. An apparatus for processing image signal data, comprising:
an interface circuit configured to receive input patch data representing a portion of a frame of image data from a source; and
a normalized cross correlation (NCC) circuit comprising:
a filtering circuit implemented in hardware and coupled to the interface circuit, the filtering circuit configured to receive the input patch data from the interface circuit and configured to perform a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data; and
a normalization circuit implemented in hardware and configured to compute a normalized score output including a plurality of normalization scores for each location of the convolution output data based on the convolution output data and the kernel data, the normalization circuit comprising a register circuit configured to store operation parameters received in a configuration cycle.

2. The apparatus of claim 1 , wherein the filtering circuit is further configured to:
compute a moving summation of the input patch data or the processed patch data, and a square of the input patch data or the processed patch data; and
output the moving summation of the input patch data or the processed patch data and the square of the input patch data or the processed patch data to the normalization circuit.

3. The apparatus of claim 1 , wherein the NCC circuit further comprises a gain offset clipping circuit configured to receive input patch data and scale the input patch data to generate the processed patch data which is a scaled version of the input patch data.

4. The apparatus of claim 1 , wherein the NCC circuit further comprises a kernel summation circuit configured to:
compute a kernel value summation and a kernel value square summation from the kernel data, wherein the kernel value summation is a sum of kernel values in each location of the kernel data and the kernel value square summation is a sum of squaring the kernel values in each location of the kernel data; and
output the kernel value summation and the kernel value square summation to the normalization circuit.

5. The apparatus of claim 1 , wherein the NCC circuit further comprises a peak finding circuit configured to select a peak value of the normalized score output from the plurality of computed normalization scores.

6. The apparatus of claim 5 , wherein the peak finding circuit is further configured to output a location of a computed normalization score in the normalized score output corresponding to the peak value.

7. The apparatus of claim 1 , wherein the NCC circuit further comprises a multiplexer configured to selectively output one of (i) the normalized score output from the normalization circuit, (ii) a scaled, offset, and bit shifted version of the convolution output data, and (iii) the received input patch data or the processed patch data derived from the input patch data according to a selection control signal as the convolution output data.

8. A method of computing a normalization score on image data from image sensors, comprising:
receiving, by an interface circuit, input patch data representing a portion of a frame of the image data from a source;
receiving, by a filtering circuit implemented in hardware of a normalized cross correlation (NCC) circuit, the input patch data from the interface circuit;
computing, by the filtering circuit of the NCC circuit, a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data;
receiving, by a normalization circuit of the NCC circuit, operation parameters in a configuration cycle;
storing, in a register circuit of the normalization circuit, the received operation parameters; and
computing, by a normalization circuit of the NCC circuit, a normalized score output including a plurality of normalization scores for each location of the convolution output data according to the stored operation parameters based on the convolution output data and the kernel data.

9. The method of claim 8 , comprising:
computing, by the filtering circuit, a moving summation of the input patch data or the processed patch data, and a square of the input patch data or the processed patch data; and
outputting, by the filtering circuit, the moving summation of the input patch data or the processed patch data and the square of the input patch data or the processed patch data to the normalization circuit.

10. The method of claim 8 , comprising:
receiving, by a gain offset clipping circuit of the NCC circuit, input patch data; and
scaling, by the gain offset clipping circuit, the input patch data to generate the processed patch data which is a scaled version of the input patch data.

11. The method of claim 8 , comprising:
computing, by a kernel summation circuit of the NCC circuit, a kernel value summation and a kernel value square summation from the kernel data, wherein the kernel value summation is a sum of kernel values in each location of the kernel data and the kernel value square summation is a sum of squaring the kernel values in each location of the kernel data; and
outputting, by the kernel summation circuit, the kernel value summation and the kernel value square summation to the normalization circuit.

12. The method of claim 8 , comprising:
selecting, by a peak finding circuit of the NCC circuit, a peak value of the normalized score output from the plurality of computed normalization scores.

13. The method of claim 12 , comprising:
outputting, by the peak finding circuit, a location of a computed normalization score in the normalized score output corresponding to the peak value.

14. The method of claim 8 , comprising:
selectively outputting, by a multiplexer of the NCC circuit, one of (i) the normalized score output from the normalization circuit, (ii) a scaled, offset, and shifted version of the convolution output data, and (iii) the received input patch data or the processed patch data derived from the input patch data according to a selection control signal as the convolution output data.

15. An electronic device, comprising:
an image sensor configured to generate image data;
an interface circuit configured to receive input patch data representing a portion of a frame of the image data from a source; and
a normalized cross correlation (NCC) circuit comprising:
a filtering circuit implemented in hardware and coupled to the interface circuit, the filtering circuit configured to receive the input patch data from the interface circuit and configured to perform a convolution on the received input patch data or processed patch data derived from the input patch data with kernel data to produce convolution output data; and
a normalization circuit implemented in hardware and configured to compute a normalized score output including a plurality of normalization scores for each location of the convolution output data based on the convolution output data and the kernel data, the normalization circuit comprising a register circuit configured to store operation parameters received in a configuration cycle.

16. The electronic device of claim 15 , wherein the filtering circuit is further configured to:
compute a moving summation of the input patch data or the processed patch data, and a square of the input patch data or the processed patch data; and
output the moving summation of the input patch data or the processed patch data and the square of the input patch data or the processed pat

Assignees

• Apple Inc

Inventors

• Wang Muge
• Sugisawa Junji

Classifications

• G06T7/32 Primary

  using correlation-based methods · CPC title

• G06V10/32

  Normalisation of the pattern dimensions · CPC title

• G06V10/7515

  Shifting the patterns to accommodate for positional errors · CPC title

• G06V10/955

  using specific electronic processors · CPC title

• G06T7/337

  involving reference images or patches · CPC title