Content-aware wide-angle images

Patent metadata

Field	Value
Publication number	US-9742994-B2
Application number	US-201313960867-A
Country	US
Kind code	B2
Filing date	Aug 7, 2013
Priority date	Aug 8, 2008
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

This specification describes technologies relating to digital images. In general, one aspect of the subject matter described in this specification can be embodied in methods that include the actions of receiving a source wide-angle image; identifying one or more locally salient features of the source wide-angle image; calculating a mapping from the source wide-angle image to a two-dimensional mapped wide-angle image according to constraints using the identified one or more spatially variable salient features; and rendering the mapped wide-angle image using the calculated mapping such that the mapped wide-angle image reduces distortion of the locally salient features relative to the distortion of the source wide angle image.

First claim

What is claimed is:

1. A method for reducing distortion in a source wide-angle image, captured using a wide-angle lens, to produce a two-dimensional mapped wide-angle image, the method comprising:
calculating, with a processor, a spatially variable mapping, from the source wide-angle image to the two-dimensional mapped wide-angle image, using an identified one or more locally salient features to be rendered undistorted in the two-dimensional mapped wide-angle image, the calculating comprising:
determining one or more weighted constraints, the weighted constraints based on the identified one or more locally salient features;
defining a linear system of descretized constraint equations as a function of local salience of the one or more locally salient features;
variably modifying the weighted constraints to solve the descretized constraint equations; and

rendering, by the processor and using the calculated spatially variable mapping, the two-dimensional mapped wide-angle image, the rendered two-dimensional mapped wide-angle image having reduced distortion relative to the source wide-angle image.

2. The method of claim 1 , further comprising:
identifying in the source wide-angle image, the one or more locally salient features.

3. The method of claim 2 , wherein identifying locally salient features includes detecting lines in the source wide-angle image.

4. The method of claim 2 , wherein identifying locally salient features includes detecting faces in the source wide-angle image.

5. The method of claim 1 , further comprising:
storing the two-dimensional mapped wide-angle image.

6. The method of claim 1 , wherein the source wide-angle image includes one or more camera properties associated with the source wide-angle image and the method further comprising:
projecting the source wide-angle image to a surface of a sphere using the one or more camera properties; and
calculating the spatially variable mapping using the projection.

7. The method of claim 1 , wherein the linear system defines conformality, line, smoothness, and regularization constraints.

8. The method of claim 1 , wherein calculating the mapping includes solving the constraint equations using a least-squares method.

9. One or more non-transitory computer-readable storage media having processor-executable instructions thereon for reducing distortion in a source wide-angle image to produce a two-dimensional mapped wide-angle image that, responsive to execution by one or more processors, performs operations comprising:
calculating a spatially variable mapping, from the source wide-angle image to the two-dimensional mapped wide-angle image, using an identified one or more locally salient features, the calculating comprising:
determining one or more weighted constraints, the weighted constraints based on the identified one or more locally salient features;
defining a linear system of descretized constraint equations as a function of local salience of the one or more locally salient features;
variably modifying the weighted constraints to solve the descretized constraint equations; and

rendering, using the calculated spatially variable mapping, the two-dimensional mapped wide-angle image.

10. The one or more non-transitory computer-readable storage media of claim 9 , the operations further comprising:
identifying, in the source wide-angle image, the one or more locally salient features to be rendered undistorted in the two-dimensional mapped wide-angle image.

11. The one or more non-transitory computer-readable storage media of claim 10 , wherein identifying locally salient features includes detecting lines in the source wide-angle image.

12. The one or more non-transitory computer-readable storage media of claim 10 , wherein identifying locally salient features includes detecting faces in the source wide-angle image.

13. The one or more non-transitory computer-readable storage media of claim 9 , the operations further comprising:
storing the two-dimensional mapped wide-angle image.

14. The one or more non-transitory computer-readable storage media of claim 9 , wherein the source wide-angle image includes the one or more camera properties associated with the source wide-angle image and the operations further comprising:
projecting the source wide-angle image to a surface of a sphere using one or more camera properties; and
calculating the spatially variable mapping using the projection.

15. The one or more non-transitory computer-readable storage media of claim 9 , wherein the linear system defines conformality, line, smoothness, and regularization constraints.

16. The one or more non-transitory computer-readable storage media of claim 9 , wherein calculating the mapping includes solving the constraint equations using a least-squares method.

17. A system comprising:
one or more processors;
memory, coupled to the one or more processors, comprising instructions for correcting distortion in a source wide-angle image to produce a two-dimensional mapped wide-angle image, the instructions executable by the one or more processors to perform:
calculating a spatially variable mapping, from a first region and a second region of the source wide-angle image to a corresponding first region and a corresponding second region of the two-dimensional mapped wide-angle image, using an identified one or more locally salient features to be rendered undistorted in each region of the two-dimensional mapped wide-angle image, the calculating for each region comprising:
determining, for each region, one or more weighted constraints, the weighted constraints based on the identified one or more locally salient features;
defining, for each region, a linear system of descretized constraint equations as a function of local salience of the one or more locally salient features;
variably modifying, for each region, the weighted constraints to solve the descretized constraint equations; and

rendering, using the calculated spatially variable mapping, the two-dimensional mapped wide-angle image.

18. The system of claim 17 , wherein the spatially variable mapping of the first region is different than the spatially variable mapping of the second region.

19. The system of claim 17 , the instructions further executable to perform:
storing the two-dimensional mapped wide-angle image.

20. The system of claim 17 , wherein the source wide-angle image includes one or more camera properties associated with the source wide-angle image and the method further comprising:
projecting the source wide-angle image to a surface of a sphere using the one or more camera properties; and
calculating the spatially variable mapping using the projection.

Assignees

• Adobe Systems Inc

Inventors

• Agarwala Aseem O
• Carroll Robert E

Classifications

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06T3/0018

  Physics · mapped topic

• H04N5/23238 Primary

  Electricity · mapped topic

• G06T5/006

  Physics · mapped topic

• G06T3/047 Primary

  Fisheye or wide-angle transformations · CPC title