Image stitching in a multi-camera array

Patent metadata

Field	Value
Publication number	US-9355433-B1
Application number	US-201514754695-A
Country	US
Kind code	B1
Filing date	Jun 30, 2015
Priority date	Jun 30, 2015
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

Images captured by multi-camera arrays with overlap regions can be stitched together using image stitching operations. An image stitching operation can be selected for use in stitching images based on a number of factors. An image stitching operation can be selected based on a view window location of a user viewing the images to be stitched together. An image stitching operation can also be selected based on a type, priority, or depth of image features located within an overlap region. Finally, an image stitching operation can be selected based on a likelihood that a particular image stitching operation will produce visible artifacts. Once a stitching operation is selected, the images corresponding to the overlap region can be stitched using the stitching operation, and the stitched image can be stored for subsequent access.

First claim

What is claimed is:

1. A method for stitching images, comprising:
accessing a set of images associated with an overlap region for display to a user, the overlap region comprising a corresponding portion of each of the set of images;
identifying one or more image features within the overlap region;
assigning a priority to each image feature;
dividing the overlap region into sub-blocks;
selecting, for each sub-block, one or more stitching operations based on the priorities assigned to the image features within the sub-block;
digitally stitching, by a hardware image processor, the set of images together to produce a stitched image using the selected stitching operations; and
storing the stitched image.

2. The method of claim 1 , wherein the set of images comprises images captured by a multi-camera array, each image in the set of images captured by a different camera in the multi-camera array.

3. The method of claim 1 , wherein assigning a priority to an image feature comprises assigning a priority based on a type of the image feature.

4. The method of claim 1 , wherein a quality of a selected stitching operation is proportional to a priority of a highest priority image feature.

5. The method of claim 1 , wherein selecting a stitching operation for a sub-block comprises selecting a stitching operation based on a highest priority image feature within the sub-block.

6. The method of claim 1 , wherein stitching the set of images together comprises, for each sub-block, stitching a portion of each of the set of images together using the stitching operation selected for the sub-block.

7. A system for stitching images, comprising:
an input configured to access a set of images associated with an overlap region for display to a user, the overlap region comprising a corresponding portion of each of the set of images;
an image processing system comprising at least one hardware image processor, the image processing system configured to:
identify one or more image features within the overlap region;
assign a priority to each image feature;
divide the overlap region into sub-blocks;
select, for each sub-block, one or more stitching operations based on the priorities assigned to the image features within the sub-block; and
digitally stitch, by the hardware image processor, the set of images together to produce a stitched image using the selected stitching operations; and

a non-transitory computer-readable storage medium configured to store the stitched image.

8. The system of claim 7 , wherein the set of images comprises images captured by a multi-camera array, each image in the set of images captured by a different camera in the multi-camera array.

9. The system of claim 7 , wherein assigning a priority to an image feature comprises assigning a priority based on a type of the image feature.

10. The system of claim 7 , wherein a quality of a selected stitching operation is proportional to a priority of a highest priority image feature.

11. The system of claim 7 , wherein selecting a stitching operation for a sub-block comprises selecting a stitching operation based on a highest priority image feature within the sub-block.

12. The system of claim 7 , wherein stitching the set of images together comprises, for each sub-block, stitching a portion of each of the set of images together using the stitching operation selected for the sub-block.

13. A non-transitory computer-readable storage medium storing executable computer instructions for stitching images, the instructions configured to, when executed by a processor, perform steps comprising:
accessing a set of images associated with an overlap region for display to a user, the overlap region comprising a corresponding portion of each of the set of images;
identifying one or more image features within the overlap region;
assigning a priority to each image feature;
dividing the overlap region into sub-blocks;
selecting, for each sub-block, one or more stitching operations based on the priorities assigned to the image features within the sub-block;
digitally stitching, by a hardware image processor, the set of images together to produce a stitched image using the selected stitching operations; and
storing the stitched image.

14. The computer-readable storage medium of claim 13 , wherein the set of images comprises images captured by a multi-camera array, each image in the set of images captured by a different camera in the multi-camera array.

15. The computer-readable storage medium of claim 13 , wherein assigning a priority to an image feature comprises assigning a priority based on a type of the image feature.

16. The computer-readable storage medium of claim 13 , wherein a quality of a selected stitching operation is proportional to a priority of a highest priority image feature.

17. The computer-readable storage medium of claim 13 , wherein selecting a stitching operation for a sub-block comprises selecting a stitching operation based on a highest priority image feature within the sub-block.

18. The computer-readable storage medium of claim 13 , wherein stitching the set of images together comprises, for each sub-block, stitching a portion of each of the set of images together using the stitching operation selected for the sub-block.

Assignees

• Gopro Inc

Inventors

• Adsumilli Balineedu Chowdary
• Campbell Scott Patrick
• Macmillan Timothy

Classifications

• G06T7/0022

  Physics · mapped topic

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T3/4038 Primary

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T3/18 Primary

  Image warping, e.g. rearranging pixels individually · CPC title