Single image rectification

Patent metadata

Field	Value
Publication number	US-9904990-B2
Application number	US-201514975644-A
Country	US
Kind code	B2
Filing date	Dec 18, 2015
Priority date	Dec 18, 2015
Publication date	Feb 27, 2018
Grant date	Feb 27, 2018

Abstract

The disclosure includes a system and method for performing image rectification using a single image and information identified from the single image. An image recognition application receives an input image, identifies a plurality of objects in the input image, estimates rectification parameters for the plurality of objects, identifies a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects, estimates final rectification parameters based on the plurality of candidate rectification parameters, computes a global transformation matrix using the final rectification parameters, and performs image rectification on the input image using the global transformation matrix.

First claim

What is claimed is:

1. A method comprising:
receiving, by one or more processors, an input image;
identifying, by the one or more processors, a plurality of objects in the input image;
estimating, by the one or more processors, a local transformation matrix for each of the plurality of objects;
calculating rectification parameters for the plurality of objects using the local transformation matrix;
identifying, by the one or more processors, a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects;
estimating, by the one or more processors, final rectification parameters based on the plurality of candidate rectification parameters;
computing, by the one or more processors, a global transformation matrix using the final rectification parameters; and
performing, by the one or more processors, image rectification on the input image using the global transformation matrix.

2. The computer-implemented method of claim 1 , wherein estimating the final rectification parameters based on the plurality of candidate rectification parameters comprises:
identifying, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and
calculating the final rectification parameters based on the set of objects.

3. The computer-implemented method of claim 1 , wherein identifying the plurality of candidate rectification parameters using the voting procedure on the rectification parameters for the plurality of objects comprises identifying the candidate rectification parameters with a maximum number of votes.

4. The computer-implemented method of claim 1 , wherein estimating the local transformation matrix for each of the plurality of objects comprises:
determining a region of interest associated with an object in the input image;
retrieving an indexed image of the object from a database;
retrieving a region of interest associated with the object in the indexed image;
mapping the region of interest of the object in the input image to the region of interest of the object in the indexed image; and
estimating the local transformation matrix based on the mapping.

5. The computer-implemented method of claim 1 , wherein the local transformation matrix is an affine transformation matrix.

6. The computer-implemented method of claim 1 , further comprising:
performing image rectification on a plurality of input images; and
performing image stitching on the rectified input images.

7. The computer-implemented method of claim 1 , further comprising performing image recognition on the rectified input image.

8. A system comprising;
one or more processors; and
a memory, the memory storing instructions, which when executed cause the one or more processor to:
receive an input image;
identify a plurality of objects in the input image;
estimate a local transformation matrix for each of the plurality of objects;
calculate rectification parameters for the plurality of objects using the local transformation matrix;
identify a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects;
estimate final rectification parameters based on the plurality of candidate rectification parameters;
compute a global transformation matrix using the final rectification parameters; and
perform image rectification on the input image using the global transformation matrix.

9. The system of claim 8 , wherein to estimate the final rectification parameters based on the plurality of candidate rectification parameters, the instructions cause the one or more processors to:
identify, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and
calculate the final rectification parameters based on the set of objects.

10. The system of claim 8 , wherein to identify the plurality of candidate rectification parameters using the voting procedure on the rectification parameters for the plurality of objects, the instructions cause the one or more processors to identify the candidate rectification parameters with a maximum number of votes.

11. The system of claim 8 , wherein to estimate the local transformation matrix for each of the plurality of objects, the instructions cause the one or more processors to:
determine a region of interest associated with an object in the input image;
retrieve an indexed image of the object from a database;
retrieve a region of interest associated with the object in the indexed image;
map the region of interest of the object in the input image to the region of interest of the object in the indexed image; and
estimate the local transformation matrix based on the mapping.

12. The system of claim 8 , wherein the local transformation matrix is an affine transformation matrix.

13. The system of claim 8 , wherein the instructions cause the one or more processors to:
perform image rectification on a plurality of input images; and
perform image stitching on the rectified input images.

14. The system of claim 8 , wherein the instructions cause the one or more processors to perform image recognition on the rectified input image.

15. A computer program product comprising a non-transitory computer readable medium storing a computer readable program, wherein the computer readable program when executed causes a computer to:
receive an input image;
identify a plurality of objects in the input image;
estimate a local transformation matrix for each of the plurality of objects;
calculate rectification parameters for the plurality of objects using the local transformation matrix;
identify a plurality of candidate rectification parameters using a voting procedure on the rectification parameters for the plurality of objects;
estimate final rectification parameters based on the plurality of candidate rectification parameters;
compute a global transformation matrix using the final rectification parameters; and
perform image rectification on the input image using the global transformation matrix.

16. The computer program product of claim 15 , wherein to estimate the final rectification parameters based on the plurality of candidate rectification parameters, the computer readable program causes the computer to:
identify, from the plurality of objects, a set of objects corresponding to the plurality of candidate rectification parameters; and
calculate the final rectification parameters based on the set of objects.

17. The computer program product of claim 15 , wherein to identify the plurality of candidate rectification parameters using the voting procedure on the rectification parameters for the plurality of objects, the computer readable program causes the computer to identify the candidate rectification parameters with a maximum number of votes.

18. The computer program product of claim 15 , wherein to estimate the local transformation matrix for each of the plurality of objects, the computer readable program causes the computer to:
determine a region of interest bordering an object in the input image;
retrieve an indexed image of the object from a database;
retrieve a region of interest bordering the object in the indexed image;
map the region of interest of the object in the input image to the region of interest of the object in the indexed image; and
estimate the local transformation matrix based on the mapping.

19. The computer program product of claim 15

Assignees

• Moraleda Jorge
• Prashnani Ekta
• Gormish Michael J
• Berkner Kathrin
• Savarese Silvio
• Ricoh Co Ltd

Inventors

• Moraleda Jorge
• Prashnani Ekta
• Gormish Michael J
• Berkner Kathrin
• Savarese Silvio

Classifications

• G06T2207/20048

  Transform domain processing · CPC title

• G06T2207/20061

  Hough transform · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T7/62

  of area, perimeter, diameter or volume · CPC title

• G06T5/10

  using non-spatial domain filtering · CPC title