Methods and apparatus for compensating for motion and/or changing light conditions during image capture

Patent metadata

Field	Value
Publication number	US-10051182-B2
Application number	US-201615286517-A
Country	US
Kind code	B2
Filing date	Oct 5, 2016
Priority date	Oct 5, 2015
Publication date	Aug 14, 2018
Grant date	Aug 14, 2018

Abstract

Methods and apparatus for compensating for motion and/or changing light conditions during image capture, e.g., in video, through use of multiple camera modules and/or images captured by multiple camera modules are described. During image capture time periods a plurality of camera modules capture images. During a first image capture time period a first camera module captures an image including a complete image of a user selected scene area of interest. During an additional image capture time period the first camera module captures an image including a portion of the scene area of interest; however, a portion of the scene area image is missing from the captured image, e.g., due to camera motion, occlusion and/or lighting conditions. Captured images from other camera modules and/or from during different image capture time periods which include the missing portion are identified and ranked; the highest ranked image is used in generating a composite image.

First claim

What is claimed is:

1. A method of generating one or more images, the method comprising:
receiving user input indicating a scene area of interest;
capturing using a first camera module, during a first image capture time period, an image including a complete image of said scene area of interest, said first camera module having a first focal length;
capturing multiple images using a set of camera modules during a second image capture time period, said set of camera modules including at least said first camera module and a second camera module, said first camera module capturing a first image including a first portion of said scene area of interest during said second image capture time period but missing a second portion of said scene area of interest during said second image capture time period, wherein said first camera module misses a portion of said scene area of interest during the second image capture time period due to a change in camera position between said first and second image capture time periods;
identifying, for use in generating a composite image, an image including said missing second portion of said scene area of interest, said identified image being: (i) an image captured by said second camera module in said set of camera modules or (ii) an image captured during a different image capture time period than said second image capture time period; and
generating the composite image from the first portion of said scene area of interest included in said first image captured by said first camera module during the second image capture time period and the identified image including said missing second portion of said scene area of interest.

2. The method of claim 1 , wherein said missing portion is an occluded portion of said scene area of interest which is occluded from a field of view of said first camera module during said second image capture time period but not during said first image capture time period.

3. A method of generating one or more images, the method comprising:
receiving user input indicating a scene area of interest;
capturing using a first camera module, during a first image capture time period, an image including a complete image of said scene area of interest, said first camera module having a first focal length;
capturing multiple images using a set of camera modules during a second image capture time period, said set of camera modules including at least said first camera module and a second camera module, said first camera module capturing a first image including a first portion of said scene area of interest during said second image capture time period but missing a second portion of said scene area of interest during said second image capture time period, wherein said first camera module misses a portion of said scene area of interest during said second image capture time period due to a change in lighting conditions between said first and second image capture time periods, said missing second portion of said scene area of interest corresponding to a portion of a first sensor which was saturated due to lighting levels higher during said second image capture time period than during said first image capture time period;
identifying, for use in generating a composite image, an image including said missing second portion of said scene area of interest, said identified image being: (i) an image captured by said second camera module in said set of camera modules or (ii) an image captured during a different image capture time period than said second image capture time period; and
generating the composite image from the first portion of said scene area of interest included in said first image captured by said first camera module during the second image capture time period and the identified image including said missing second portion of said scene area of interest.

4. A method of generating one or more images, the method comprising:
capturing multiple images using a set of camera modules during a second image capture time period, said set of camera modules including at least a first camera module and a second camera module, said first camera module capturing a first image including a first portion of a scene area of interest during said second image capture time period but missing a second portion of said scene area of interest during said second image capture time period;
identifying, for use in generating a composite image, an image including said missing second portion of said scene area of interest, said identified image being: (i) an image captured by said second camera module in said set of camera modules or (ii) an image captured during a different image capture time period than said second image capture time period, wherein identifying, for use in generating a composite image, said image including said missing second portion of said scene area of interest includes:
i) identifying one or more images including said second missing portion; and
ii) ranking the identified one or more images including said missing second portion of said scene area of interest based on at least one of resolution or time of image capture; and
iii) selecting, based on the ranking of the identified one or more images including said missing second portion, a highest ranked one of the identified one or more images including said second missing portion for use as the identified image for use in combining with the first image captured by the first camera module; and

generating the composite image from the first portion of said scene area of interest included in said first image captured by said first camera module during the second image capture time period and the identified image including said missing second portion of said scene area of interest.

5. The method of claim 4 , wherein ranking the identified one or more images including the second missing portion includes weighting the identified one or more images on a per image basis based on the resolution of the image being ranked.

6. The method of claim 5 , wherein ranking the identified one or more images including the second missing portion includes weighting images based on image resolution, higher resolution images being weighted more heavily than lower resolution images.

7. The method of claim 6 , wherein ranking the identified one or more images including the second missing portion includes weighting images based on temporal proximity to the second image capture time period, images captured during the second image capture time period weighted more heavily than images captured during another image capture time period.

8. The method of claim 6 , wherein ranking the identified one or more images includes weighting images based on temporal proximity to the second image capture time period, images captured during an image capture time period immediately preceding or subsequent to said second image capture time period being weighted more heavily than images captured in a time period two or more image capture time periods away from said second image capture time period.

9. The method of claim 8 , wherein when an image corresponding to a different image capture time period and an image corresponding to the second image capture time period are identified as including the second missing scene portion, ranking the identified one or more images includes:
determining the resolution of the image captured during the second image capture time period including the second missing scene portion;
determining the resolution of the image captured during a different image capture time period including the second missing scene portion; and
when the resolution of the image captured during a different image capture time period including the second missing scene portion is higher than the resolutio

Assignees

• Light Labs Inc

Inventors

• Pulli Kari
• Shroff Nitesh
• Shroff Sapna A

Classifications

• H04N23/682 Primary

  Vibration or motion blur correction · CPC title

• H04N23/57

  Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices · CPC title

• H04N5/265 Primary

  Mixing · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title