Compensating for geometric distortion of images in constrained processing environments

Patent metadata

Field	Value
Publication number	US-11348209-B2
Application number	US-202017027287-A
Country	US
Kind code	B2
Filing date	Sep 21, 2020
Priority date	May 5, 2016
Publication date	May 31, 2022
Grant date	May 31, 2022

Abstract

An image processing method determines a geometric transform of a suspect image by efficiently evaluating a large number of geometric transform candidates in environments with limited processing resources. Processing resources are conserved by using complementary methods for determining a geometric transform of an embedded signal. One method excels at higher geometric distortion, and specifically, distortion caused by greater tilt angle of a camera. Another method excels at lower geometric distortion, for weaker signals. Together, the methods provide a more reliable detector of an embedded data signal in image across a larger range of distortion while making efficient use of limited processing resources in mobile devices.

First claim

I claim:

1. A method of reading an embedded digital payload in an image, the method comprising:
obtaining a suspect image;
starting with seed geometric transform candidates, grouping the seed geometric transform candidates into groups based on proximity to each other in geometric parameter space;
in plural refinement stages, refining the seed geometric transform candidates in each group to select a subset of geometric transform candidates in a group to pass to a subsequent refinement stage for each group;
performing an iterative process to refine the subset of geometric transform candidates in each group, the iterative process finding updated geometric transform candidates based on how the updated geometric transform candidates improve detection metrics for an embedded signal in the suspect image;
selecting a first geometric transform candidate from among the groups after plural refinement stages; and
using the first geometric transform candidate to extract a digital payload from the suspect image.

2. The method of claim 1 wherein at least one of the plural refinement stages applies a fitting process that produces first refined geometric transform candidates having detection metrics for an embedded signal in an image feature space that satisfy predetermined criteria.

3. The method of claim 2 wherein the fitting process comprises:
a) obtaining transformed coordinates of reference signal components, the transformed coordinates having been geometrically transformed by a geometric transform candidate;
b) for the reference signal components, determining updated coordinates by locating an image feature in a neighborhood in the suspect image around the transformed coordinates of reference signal components, the image feature corresponding to a potential reference signal component in the suspect image; and
c) determining a new geometric transform that provides a least squares mapping between coordinates of the reference signal components and the updated coordinates.

4. The method of claim 3 wherein the reference signal components comprise peaks in the image feature space.

5. The method of claim 4 wherein the image feature space comprises a spatial frequency transform domain.

6. The method of claim 1 wherein the seed geometric transform candidates comprise candidates representing bearing and camera tilt angles, and the proximity is based on proximity in spatial scale.

7. A reader device comprising:
an imager operable to capture an image;
memory configured to store the image from the imager;
a processor configured with instructions to perform the following acts to extract a digital payload from the image in the memory:
in plural refinement stages, refine groups of seed geometric transform candidates in each group to select a subset of geometric transform candidates in a group to pass to a subsequent refinement stage for each group, the seed geometric transform candidates being organized into groups based on proximity to each other in geometric parameter space;
perform an iterative process to refine the subset of geometric transform candidates in each group, the iterative process finding updated geometric transform candidates based on how the updated geometric transform candidates improve detection metrics for an embedded signal in the image;
select a first geometric transform candidate from among the groups after plural refinement stages; and
use the first geometric transform candidate to extract a digital payload from the image.

8. The reader device of claim 7 wherein at least one of the plural refinement stages applies a fitting process that produces first refined geometric transform candidates having detection metrics for an embedded signal in an image feature space that satisfy predetermined criteria.

9. The reader device of claim 8 wherein the processor is configured with instructions to:
a) obtain transformed coordinates of reference signal components, the transformed coordinates having been geometrically transformed by a geometric transform candidate;
b) for the reference signal components, determine updated coordinates by locating an image feature in a neighborhood in the suspect image around the transformed coordinates of reference signal components, the image feature corresponding to a potential reference signal component in the image; and
c) determine a new geometric transform that provides a least squares mapping between coordinates of the reference signal components and the updated coordinates.

10. The reader device of claim 9 wherein the reference signal components comprise peaks in the image feature space.

11. The reader device of claim 10 wherein the image feature space comprises a spatial frequency transform domain.

12. The reader device of claim 7 wherein the seed geometric transform candidates comprise candidates representing bearing and camera tilt angles, and the proximity is based on proximity in spatial scale.

13. The reader device of claim 7 wherein the seed geometric transform candidates are grouped by spatial scale.

14. A non-transitory computer readable medium on which is stored instructions, which when executed by a processor, perform a method of reading an embedded digital payload in an image, the method comprising:
obtaining a suspect image;
starting with seed geometric transform candidates, grouping the seed geometric transform candidates into groups based on proximity to each other in geometric parameter space;
in plural refinement stages, refining the seed geometric transform candidates in each group to select a subset of geometric transform candidates in a group to pass to a subsequent refinement stage for each group;
performing an iterative process to refine the subset of geometric transform candidates in each group, the iterative process finding updated geometric transform candidates based on how the updated geometric transform candidates improve detection metrics for an embedded signal in the suspect image;
selecting a first geometric transform candidate from among the groups after plural refinement stages; and
using the first geometric transform candidate to extract a digital payload from the suspect image.

15. The computer readable medium of claim 14 wherein at least one of the plural refinement stages applies a fitting process that produces first refined geometric transform candidates having detection metrics for an embedded signal in an image feature space that satisfy predetermined criteria.

16. The computer readable medium of claim 15 wherein the fitting process comprises:
a) obtaining transformed coordinates of reference signal components, the transformed coordinates having been geometrically transformed by a geometric transform candidate;
b) for the reference signal components, determining updated coordinates by locating an image feature in a neighborhood in the suspect image around the transformed coordinates of reference signal components, the image feature corresponding to a potential reference signal component in the suspect image; and
c) determining a new geometric transform that provides a least squares mapping between coordinates of the reference signal components and the updated coordinates.

17. The computer readable medium of 36 wherein the reference signal components comprise peaks in the image feature space.

18. The computer readable medium of claim 17 wherein the image feature space comprises a spatial frequency transform domain.

19. The computer readable medium of claim 14 wherein the seed geometric transform candidates comprise candidates re

Assignees

• Digimarc Corp

Inventors

• Holub Vojtech

Classifications

• G06T2201/0052

  Embedding of the watermark in the frequency domain · CPC title

• G06T2201/0601

  whereby calibration information is embedded in the watermark, e.g. a grid, a scale, a list of transformations · CPC title

• G06T2201/0061

  Embedding of the watermark in each block of the image, e.g. segmented watermarking · CPC title

• G06T3/4084

  in the transform domain, e.g. fast Fourier transform [FFT] domain scaling · CPC title

• G06T1/0064 Primary

  Geometric transfor invariant watermarking, e.g. affine transform invariant · CPC title