Systems and methods for content-adaptive image stitching

Patent metadata

Field	Value
Publication number	US-10373360-B2
Application number	US-201715447827-A
Country	US
Kind code	B2
Filing date	Mar 2, 2017
Priority date	Mar 2, 2017
Publication date	Aug 6, 2019
Grant date	Aug 6, 2019

Abstract

A method for stitching images by an electronic device is described. The method includes obtaining at least two images. The method also includes selecting a stitching scheme from a set of stitching schemes based on one or more content measures of the at least two images. The set of stitching schemes includes a first stitching scheme, a second stitching scheme, and a third stitching scheme. The method further includes stitching the at least two images based on a selected stitching scheme.

First claim

What is claimed is:

1. A method for stitching images by an electronic device, comprising:
obtaining at least two images;
selecting a stitching scheme from a set of stitching schemes based on one or more content measures including an object coverage measure that indicates a span of an object in at least a portion of at least one of the at least two images, wherein the set of stitching schemes includes a first stitching scheme, a second stitching scheme, and a third stitching scheme; and
stitching the at least two images based on a selected stitching scheme.

2. The method of claim 1 , wherein stitching the at least two images produces a surround view image of a scene.

3. The method of claim 1 , wherein the at least two images are fisheye images, and wherein the method further comprises transforming overlapping outer regions of the at least two images into rectangular images using a geometrical transformation.

4. The method of claim 1 , further comprising determining sub-regions of an overlapping region of the at least two images based on the one or more content measures.

5. The method of claim 4 , further comprising performing selecting the stitching scheme for each of the sub-regions.

6. The method of claim 5 , wherein different stitching schemes are selected for at least two of the sub-regions.

7. The method of claim 1 , wherein the one or more content measures further comprise a motion measure.

8. The method of claim 1 , wherein the first stitching scheme is static seam-based stitching, the second stitching scheme is dynamic seam-based stitching, and the third stitching scheme is dynamic warp stitching.

9. The method of claim 8 , wherein selecting the stitching scheme comprises:
selecting static seam-based stitching in a case that a motion measure is greater than a motion threshold;
determining whether a match is unreliable in a case that the motion measure is not greater than the motion threshold; and
selecting static seam-based stitching in a case that the match is unreliable.

10. The method of claim 8 , wherein selecting the stitching scheme comprises selecting dynamic warp stitching in a case that a disparity measure is greater than a disparity threshold or the object coverage measure meets a coverage criterion.

11. The method of claim 8 , wherein selecting the stitching scheme comprises selecting dynamic seam-based stitching in a case that a disparity measure is not greater than a disparity threshold and the object coverage measure does not meet a coverage criterion.

12. The method of claim 1 , further comprising:
selecting a blending region based on at least one of a motion measure or a disparity measure; and
blending the at least two images in the blending region.

13. The method of claim 1 , wherein stitching the at least two images comprises performing color correction based on a disparity measure.

14. The method of claim 1 , further comprising constraining a dynamic seam to align at a sub-region boundary in a case that the selected stitching scheme is dynamic seam-based stitching.

15. An electronic device for stitching images, comprising:
a memory;
a processor coupled to the memory, wherein the processor is configured to:
obtain at least two images;
select a stitching scheme from a set of stitching schemes based on one or more content measures including an object coverage measure that indicates a span of an object in at least a portion of at least one of the at least two images, wherein the set of stitching schemes includes a first stitching scheme, a second stitching scheme, and a third stitching scheme; and
stitch the at least two images based on a selected stitching scheme.

16. The electronic device of claim 15 , wherein stitching the at least two images produces a surround view image of a scene.

17. The electronic device of claim 16 , wherein the at least two images are fisheye images, and wherein the processor is configured to transform overlapping outer regions of the at least two images into rectangular images using a geometrical transformation.

18. The electronic device of claim 15 , wherein the processor is configured to determine sub-regions of an overlapping region of the at least two images based on the one or more content measures.

19. The electronic device of claim 18 , wherein the processor is configured to perform selecting the stitching scheme for each of the sub-regions.

20. The electronic device of claim 19 , wherein different stitching schemes are selected for at least two of the sub-regions.

21. The electronic device of claim 15 , wherein the one or more content measures further comprise a motion measure.

22. The electronic device of claim 15 , wherein the first stitching scheme is static seam-based stitching, the second stitching scheme is dynamic seam-based stitching, and the third stitching scheme is dynamic warp stitching.

23. The electronic device of claim 22 , wherein the processor is configured to select the stitching scheme by:
selecting static seam-based stitching in a case that a motion measure is greater than a motion threshold;
determining whether a match is unreliable in a case that the motion measure is not greater than the motion threshold; and
selecting static seam-based stitching in a case that the match is unreliable.

24. The electronic device of claim 22 , wherein the processor is configured to select the stitching scheme by selecting dynamic warp stitching in a case that a disparity measure is greater than a disparity threshold or the object coverage measure meets a coverage criterion.

25. The electronic device of claim 22 , wherein the processor is configured to select the stitching scheme by selecting dynamic seam-based stitching in a case that a disparity measure is not greater than a disparity threshold and the object coverage measure does not meet a coverage criterion.

26. The electronic device of claim 15 , wherein the processor is configured to:
select a blending region based on at least one of a motion measure or a disparity measure; and
blend the at least two images in the blending region.

27. The electronic device of claim 15 , wherein the processor is configured to stitch the at least two images by performing color correction based on a disparity measure.

28. The electronic device of claim 15 , wherein the processor is configured to constrain a dynamic seam to align at a sub-region boundary in a case that the selected stitching scheme is dynamic seam-based stitching.

29. A non-transitory tangible computer-readable medium storing computer executable code, comprising:
code for causing an electronic device to obtain at least two images;
code for causing the electronic device to select a stitching scheme from a set of stitching schemes based on one or more content measures including an object coverage measure that indicates a span of an object in at least a portion of at least one of the at least two images, wherein the set of stitching schemes includes a first stitching scheme, a second stitching scheme, and a third stitching scheme; and
code for causing the electronic device to stitch the at least two images based on a selected stitching scheme.

30. The computer-readable medium of claim 29 , further comprising code for causing the electronic device to determine sub-regions of overlapping regions of the at least two images based on

Assignees

• Qualcomm Inc

Inventors

• Gorur Sheshagiri Pushkar
• Choudhuri Chiranjib
• Banerjee Sudipto
• Gupte Ajit Deepak
• Baheti Pawan Kumar
• Rao Ajit Venkat

Classifications

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T11/60 Primary

  Creating or editing images; Combining images with text · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T3/0081

  Physics · mapped topic

• G06F16/444

  Spatial browsing, e.g. 2D maps, 3D or virtual spaces · CPC title