Systems and methods for selecting an image transform

Patent metadata

Field	Value
Publication number	US-10157439-B2
Application number	US-201614991871-A
Country	US
Kind code	B2
Filing date	Jan 8, 2016
Priority date	Jul 20, 2015
Publication date	Dec 18, 2018
Grant date	Dec 18, 2018

Abstract

An electronic device for selecting a transform is described. The electronic device includes at least one image sensor, a memory, and a processor coupled to the memory and to the at least one image sensor. The processor is configured to obtain at least two images from the at least one image sensor. The processor is also configured to characterize structural content of each of the at least two images to produce a characterization for each image that is relevant to transform performance. The processor is further configured to select at least one transform from a set of transforms based on the characterization. The processor is additionally configured to apply the at least one transform to at least one of the images to substantially align the at least two images.

First claim

What is claimed is:

1. An electronic device for selecting a transform, comprising:
at least one image sensor;
a memory;
a processor coupled to the memory and to the at least one image sensor, wherein the processor is configured to:
obtain at least two images from the at least one image sensor;
characterize structural content of each of the at least two images to produce a characterization for each image that quantifies the structural content, wherein the processor is configured to extract keypoints from at least one of the images;
select at least one transform from a set of transforms based on at least one characterization by determining whether a number of the extracted keypoints meets a keypoint number criterion and whether a keypoint distribution of the extracted keypoints meets a keypoint distribution criterion; and
apply the at least one transform to at least one of the images to substantially align the at least two images.

2. The electronic device of claim 1 , wherein the processor is configured to characterize the structural content by determining one or more metrics, wherein the one or more metrics comprise at least one of a spatial response, structure pervasiveness, spatial distribution, or transform response.

3. The electronic device of claim 2 , wherein the processor is configured to compare the one or more metrics to one or more thresholds.

4. The electronic device of claim 1 , wherein the set of transforms is arranged in a hierarchy of transforms, and wherein the processor is configured to select the at least one transform based on the hierarchy.

5. The electronic device of claim 4 , wherein the processor is configured to determine whether to select the at least one transform proceeding in a hierarchical order, wherein the hierarchy is ordered by a number of transform parameters associated with each of the transforms in the set of transforms.

6. The electronic device of claim 1 , wherein the processor is configured to characterize the structural content by at least one of determining a number of structures, determining a spatial distribution of structures, or determining a transform response.

7. The electronic device of claim 1 , wherein the processor is configured to perform at least one of registering, rectifying, or warping the at least one of the at least two images based on the at least one transform.

8. The electronic device of claim 7 , wherein the processor is configured to compute a reprojection error.

9. The electronic device of claim 1 , further comprising at least two lenses, wherein the processor is configured to switch between the at least two lenses based on the image alignment.

10. The electronic device of claim 1 , wherein the set of transforms comprises: a transform including scale, rotation, and x-y shift; a transform including scale and x-y shift; a transform including x-y shift; a transform including x shift; and a transform including y shift.

11. The electronic device of claim 1 , wherein the processor is configured to select a transform based on a look-up table of application accuracy requirements.

12. A method for selecting a transform performed by an electronic device, comprising:
obtaining at least two images;
characterizing structural content of each of the at least two images to produce a characterization for each image that quantifies the structural content, comprising extracting keypoints from at least one of the images;
selecting at least one transform from a set of transforms based on at least one characterization by determining whether a number of the extracted keypoints meets a keypoint number criterion and whether a keypoint distribution of the extracted keypoints meets a keypoint distribution criterion; and
applying the at least one transform to at least one of the images to substantially align the at least two images.

13. The method of claim 12 , wherein characterizing the structural content comprises determining one or more metrics, wherein the one or more metrics comprise at least one of a spatial response, structure pervasiveness, spatial distribution, or transform response.

14. The method of claim 13 , wherein selecting the at least one transform comprises comparing the one or more metrics to one or more thresholds.

15. The method of claim 12 , wherein the set of transforms is arranged in a hierarchy of transforms, and wherein selecting the at least one transform is based on the hierarchy.

16. The method of claim 15 , wherein selecting the at least one transform comprises determining whether to select the at least one transform proceeding in a hierarchical order, wherein the hierarchy is ordered by a number of transform parameters associated with each of the transforms in the set of transforms.

17. The method of claim 12 , wherein characterizing the structural content comprises at least one of determining a number of structures, determining a spatial distribution of structures, or determining a transform response.

18. The method of claim 12 , further comprising performing at least one of registering, rectifying, or warping the at least one of the at least two images based on the at least one transform.

19. The method of claim 18 , further comprising computing a reprojection error.

20. The method of claim 12 , further comprising switching between at least two lenses based on the image alignment.

21. The method of claim 12 , wherein the set of transforms comprises: a transform including scale, rotation, and x-y shift; a transform including scale and x-y shift; a transform including x-y shift; a transform including x shift; and a transform including y shift.

22. The method of claim 12 , further comprising selecting a transform based on a look-up table of application accuracy requirements.

23. A computer-program product for selecting a transform, comprising a non-transitory tangible computer-readable medium having instructions thereon, the instructions comprising:
code for causing an electronic device to obtain at least two images;
code for causing the electronic device to characterize structural content of each of the at least two images to produce a characterization for each image that quantifies the structural content, comprising code for causing the electronic device to extract keypoints from at least one of the images;
code for causing the electronic device to select at least one transform from a set of transforms based on at least one characterization comprising code for causing the electronic device to determine whether a number of the extracted keypoints meets a keypoint number criterion and whether a keypoint distribution of the extracted keypoints meets a keypoint distribution criterion; and
code for causing the electronic device to apply the at least one transform to at least one of the images to substantially align the at least two images.

24. The computer-program product of claim 23 , wherein the code for causing the electronic device to characterize the structural content comprises code for causing the electronic device to determine one or more metrics, wherein the one or more metrics comprise at least one of a spatial response, structure pervasiveness, spatial distribution, or transform response.

25. The computer-program product of claim 23 , wherein the set of transforms is arranged in a hierarchy of transforms, and wherein the code for causing the electronic device to select the at least one transform is based on the hierarchy.

Assignees

• Qualcomm Inc

Inventors

• Nash James
• Atanassov Kalin Mitkov
• Siddiqui Hasib Ahmed
• Lindner Albrecht Johannes

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• H04N23/69

  Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06T1/20 Primary

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• H04N5/232

  Electricity · mapped topic