Image correlation processing by addition of reaggregation

Patent metadata

Field	Value
Publication number	US-12597147-B2
Application number	US-202318302252-A
Country	US
Kind code	B2
Filing date	Apr 18, 2023
Priority date	Jun 28, 2022
Publication date	Apr 7, 2026
Grant date	Apr 7, 2026

Abstract

A method of image correlation includes: obtaining a first image frame with a set of distinct features; obtaining a second image frame at least partially overlapping the first image frame; carrying out a cross-correlation operation of the first and second image frames to obtain a cross-correlation result indicating correlation values for different offset positions of the second image frame with respect to the first image frame, the highest correlation value with its associated offset position in the cross-correlation result indicating the most likely offset position of the second image frame with respect to the first image frame; and carrying out a reaggregation operation of the cross-correlation result to generate a distinct global maximum correlation value from a plurality of non-distinct correlation values in an aggregated cross-correlation result. The reaggregation operation involves carrying out a convolution of the cross-correlation result with a 1×2, 2×1, or 2×2 array of 1's.

First claim

The invention claimed is:

1 . A computer-implemented method of image correlation, the method comprising:
obtaining a first image frame with a set of distinct features; obtaining a second image frame at least partially overlapping the first image frame such that at least some of the features of the first image frame are visible in the second image frame, the first and second image frames being obtained by an image sensor comprising an array of pixels; carrying out a cross-correlation operation of the first and second image frames to obtain a cross-correlation result indicating correlation values for different offset positions of the second image frame with respect to the first image frame, the highest correlation value with its associated offset position in the cross-correlation result indicating the most likely offset position of the second image frame with respect to the first image frame; carrying out a reaggregation operation of the cross-correlation result to generate a distinct global maximum correlation value from a plurality of non-distinct correlation values in a reaggregated cross-correlation result, wherein the reaggregation operation involves carrying out a convolution of the cross-correlation result with a 1×2, 2×1, or 2×2 array of 1′s, said convolution being carried out over every 2×2 or 1×2 or 2×1 grouping in the cross-correlation to produce a new array of values; and correcting a mouse pointer position of an optical mouse based on the distinct global maximum correlation value generated by carrying out the reaggregation operation of the cross-correlation results.

2 . The method according to claim 1 , wherein the convolution involves summing every overlapping 2×2 group of values into a new array of reaggregated values that sit spatially at the center of each 2×2 group of raw values before reaggregation of the cross-correlation result.

3 . The method according to claim 1 , wherein the value of the distinct global maximum correlation value equals or substantially equals the number of distinct features in the first and/or second image frames, when digitizing the analog value that lower analog values are assigned larger digital values, indicating more significant features from a greater reduction in light seen by a given pixel.

4 . The method according to claim 1 , wherein the method further comprises determining spatial displacement of the second image frame with respect to the first image frame from the position of the distinct global maximum correlation value in the aggregated cross-correlation result.

5 . The method according to claim 4 , wherein the spatial displacement is a two-dimensional value.

6 . The method according to claim 4 , wherein the method further comprises taking the integer value of the location of the distinct global maximum correlation value in the reaggregated cross-correlation result as macropixel displacement of the second image frame with respect to the first image frame.

7 . The method according to claim 4 , wherein the method further comprises performing quadratic interpolation on a 3×3, 3×4, 4×4, or 4×4 array of correlation values centered on the distinct global maximum correlation value of the reaggregated cross-correlation result, and taking the result of the quadratic interpolation as subpixel displacement of the second image frame ( 3 ) with respect to the first image frame.

8 . The method according to claim 4 , wherein the method further comprises subtracting a value of (0.5, 0.5) from the spatial displacement to obtain final spatial displacement, and including the final spatial displacement in a displacement report.

9 . The method according to claim 4 , wherein no correction for quadratic distortion caused by imposing a quadratic fit to interpolate a linear distribution, is made when determining the spatial displacement.

10 . The method according to claim 1 , wherein the image correlation method is used to track spatial displacement of an optical computer mouse.

11 . The method according to claim 10 , wherein the second image frame is obtained after a given time duration after the first image frame, and wherein the time duration is dependent on the movement speed of the mouse.

12 . The method according to claim 1 , wherein the first image frame is the immediately previous image frame to the second image frame.

13 . The method according to claim 1 , wherein the steps of obtaining the first and second image frames, the cross-correlation step, and the reaggregation step are carried out by an imaging apparatus, and in particular an optical computer mouse.

14 . A non-transitory computer program product comprising instructions for implementing the steps of the method according to claim 1 when loaded and run on computing means of a computing device.

15 . An apparatus for carrying out an image correlation process, the apparatus comprising means for:
obtaining a first image frame with a set of distinct features; obtaining a second image frame at least partially overlapping the first image frame such that at least some of the features of the first image frame are visible in the second image frame, the first and second image frames being obtained by an image sensor comprising an array of pixels; carrying out a cross-correlation operation of the first and second image frames to obtain a cross-correlation result indicating correlation values for different offset positions of the second image frame with respect to the first image frame, the highest correlation value with its associated offset position in the cross-correlation result indicating the most likely offset position of the second image frame with respect to the first image frame; carrying out a reaggregation operation of the cross-correlation result to generate a distinct global maximum correlation value from a plurality of non-distinct correlation values in a reaggregated cross-correlation result, wherein the reaggregation operation involves carrying out a convolution of the cross-correlation result with a 1×2, 2×1, or 2×2 array of 1's; and correcting a mouse pointer position of an optical mouse based on the distinct global maximum correlation value generated by carrying out the reaggregation operation of the cross-correlation results.

Assignees

• Em Microelectronic Marin Sa

Inventors

• Moore Brian

Classifications

• G06F3/03543

  Mice or pucks (G06F3/03541 takes precedence) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/32 Primary

  using correlation-based methods · CPC title