Conversion between aspect ratios in camera

Patent metadata

Field	Value
Publication number	US-9674429-B2
Application number	US-201615011253-A
Country	US
Kind code	B2
Filing date	Jan 29, 2016
Priority date	Aug 22, 2013
Publication date	Jun 6, 2017
Grant date	Jun 6, 2017

Abstract

A camera system captures an image in a source aspect ratio and applies a transformation to the input image to scale and warp the image to generate an output image having a target aspect ratio different than the source aspect ratio. The output image has the same field of view as the input image, maintains image resolution, and limits distortion to levels that do not substantially affect the viewing experience. In one embodiment, the output image is non-linearly warped relative to the input image such that a distortion in the output image relative to the input image is greater in a corner region of the output image than a center region of the output image.

First claim

The invention claimed is:

1. A method for converting between aspect ratios, comprising:
obtaining an input image having a source aspect ratio;
applying, by a processing device, a transformation to the input image to scale and warp the input image to generate an output image having a target aspect ratio, the target aspect ratio different than the source aspect ratio, the output image non-linearly warped relative to the input image such that a distortion in the output image relative to the input image is greater in a corner region of the output image than a center region of the output image, wherein applying the transformation comprises shifting pixels of the input image by an amount represented by a non-linear function of original pixel positions of the pixels, wherein applying the transformation to the input image to scale and warp the input image comprises:
applying a scaling function to the input image to linearly stretch or compress the input image to generate a scaled image having the target aspect ratio; and
applying a non-linear warping function to the scaled image to reduce distortion of a center region of the scaled image relative to the input image and increase distortion of a corner region of the scaled image relative to the input image; and

outputting the output image having the target aspect ratio.

2. The method of claim 1 , wherein applying the transformation comprises applying a combined function that performs both warping and scaling.

3. The method of claim 1 , wherein the processing device comprises a camera.

4. The method of claim 1 , wherein the output image has a same field of view as the input image.

5. The method of claim 1 , wherein the source aspect ratio comprises a 4:3 aspect ratio and the target aspect ratio comprises a 16:9 aspect ratio.

6. A method for converting between aspect ratios, comprising:
obtaining an input image having a source aspect ratio;
applying, by a processing device, a transformation to the input image to scale and warp the input image to generate an output image having a target aspect ratio, the target aspect ratio different than the source aspect ratio, the output image non-linearly warped relative to the input image such that a distortion in the output image relative to the input image is greater in a corner region of the output image than a center region of the output image, wherein applying the transformation comprises shifting pixels of the input image by an amount represented by a non-linear function of original pixel positions of the pixels, wherein applying the transformation to the input image to scale and warp the input image comprises:
applying a non-linear warping function to the input image to generate a warped image, the warped image having a reduced distortion of a center region of the warped image relative to the input image and an increased distortion of a corner region of the warped image relative to the input image; and
applying a scaling function to the warped image to linearly stretch or compress the warped image to generate the output image having the target aspect ratio; and

outputting the output image having the target aspect ratio.

7. The method of claim 6 , wherein applying the transformation comprises applying a combined function that performs both warping and scaling.

8. The method of claim 6 , wherein the output image has a same field of view as the input image.

9. The method of claim 6 , wherein the source aspect ratio comprises a 4:3 aspect ratio and the target aspect ratio comprises a 16:9 aspect ratio.

10. The method of claim 6 , wherein the processing device comprises a camera.

11. A computing apparatus comprising:
an input device to obtain an input image having a source aspect ratio;
a processor to process the input image captured by the image sensor; and
a non-transitory computer-readable storage medium storing instructions that when executed by the processor cause the processor to perform steps including:
applying a transformation to the input image to scale and warp the input image to generate an output image having a target aspect ratio, the target aspect ratio different than the source aspect ratio, the output image non-linearly warped relative to the input image such that a distortion in the output image relative to the input image is greater in a corner region of the output image than a center region of the output image, wherein applying the transformation comprises shifting pixels of the input image by an amount represented by a non-linear function of original pixel positions of the pixels, wherein applying the transformation to the input image to scale and warp the input image comprises:
applying a scaling function to the input image to linearly stretch or compress the input image to generate a scaled image having the target aspect ratio; and
applying a non-linear warping function to the scaled image to reduce distortion of a center region of the scaled image relative to the input image and increase distortion of a corner region of the scaled image relative to the input image; and

an output device to output the output image having the target aspect ratio.

12. The computing apparatus of claim 11 , wherein applying the transformation comprising applying a combined function that performs both warping and scaling.

13. The computing apparatus of claim 11 , wherein the computing apparatus comprises a camera.

14. The computing apparatus of claim 11 , wherein the output image has a same field of view as the input image.

15. The computing apparatus of claim 11 , wherein the source aspect ratio comprises a 4:3 aspect ratio and the target aspect ratio comprises a 16:9 aspect ratio.

16. A computing apparatus comprising:
an input device to obtain an input image having a source aspect ratio;
a processor to process the input image captured by the image sensor; and
a non-transitory computer-readable storage medium storing instructions that when executed by the processor cause the processor to perform steps including:
applying a transformation to the input image to scale and warp the input image to generate an output image having a target aspect ratio, the target aspect ratio different than the source aspect ratio, the output image non-linearly warped relative to the input image such that a distortion in the output image relative to the input image is greater in a corner region of the output image than a center region of the output image, wherein applying the transformation comprises shifting pixels of the input image by an amount represented by a non-linear function of original pixel positions of the pixels, wherein applying the transformation to the input image to scale and warp the input image comprises:
applying a non-linear warping function to the input image to generate a warped image, the warped image having a reduced distortion of a center region of the warped image relative to the input image and an increased distortion of a corner region of the warped image relative to the input image;
applying a scaling function to the warped image to linearly stretch or compress the warped image to generate the output image having the target aspect ratio; and

an output device to output the output image having the target aspect ratio.

17. The computing apparatus of claim 16 , wherein applying the transformation comprises applying a combined function that performs both warping and scaling.

18. The computing apparatus of claim 16 , wherein the computing apparatus comprises a camera.

19. The computing apparatus of claim 16 , wherein the output im

Assignees

• Gopro Inc

Inventors

• Woodman Nicholas D
• Chawla Sumit
• Segapelli Loic
• Campbell Scott Patrick

Classifications

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• G09G5/005 Primary

  Adapting incoming signals to the display format of the display terminal · CPC title

• G06T3/4084

  in the transform domain, e.g. fast Fourier transform [FFT] domain scaling · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• H04N7/0122

  the input and the output signals having different aspect ratios · CPC title