Image capture device with contemporaneous image correction mechanism

Patent metadata

Field	Value
Publication number	US-10733472-B2
Application number	US-201816221888-A
Country	US
Kind code	B2
Filing date	Dec 17, 2018
Priority date	Jun 21, 2007
Publication date	Aug 4, 2020
Grant date	Aug 4, 2020

Abstract

A hand-held or otherwise portable or spatial or temporal performance-based image capture device includes one or more lenses, an aperture and a main sensor for capturing an original main image. A secondary sensor and optical system are for capturing a reference image that has temporal and spatial overlap with the original image. The device performs an image processing method including capturing the main image with the main sensor and the reference image with the secondary sensor, and utilizing information from the reference image to enhance the main image. The main and secondary sensors are contained together within a housing.

First claim

What is claimed is:

1. An electronic device comprising:
one or more processors; and
one or more computer-readable media storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving first image data representing a first image;
receiving second image data representing a second image;
determining a first region of the first image;
determining a second region of the second image;
determining a difference between a Discrete Cosine Transform (DCT) coefficient of the first region and a DCT coefficient of the second region;
based at least in part on the difference, assigning the first region of the first image as either a background or a foreground and assigning the second region of the second image as the other of the background or the foreground; and
correcting a defect in the first image or the second image based at least in part on whether the first image is assigned as the background or the foreground.

2. The electronic device as recited in claim 1 , the operations further comprising:
determining that the first region of the first image comprises a first depiction of at least a portion of a face; and
determining that the second region of the second image comprises a second depiction of at least the portion of the face.

3. The electronic device as recited in claim 1 , the operations further comprising:
generating third image data representing a third image by at least replacing the first region of the first image with the second region of the second image;
receiving fourth image data representing a fourth image;
determining a third region of the fourth image; and
based at least in part on a difference between the second region and the third region, selecting the second region of the second image for generating the third image data.

4. The electronic device as recited in claim 1 , the operations further comprising:
determining that the second region comprises a more desirable depiction of an object than a depiction of the object in the first region; and
determining to replace the first region of the first image with the second region of the second image based at least in part on the second region comprising the more desirable depiction of the object relative to the first region.

5. The electronic device as recited in claim 1 , the operations further comprising:
determining that the first region is associated with a sensor defect; and
determining that the second region is not associated with the sensor defect,
and wherein correcting the defect in the first image or the second image is further based at least in part on the first region being associated with the sensor defect and the second region not being associated with the sensor defect.

6. The electronic device as recited in claim 1 , the operations further comprising:
storing fourth image data representing a fourth image, the fourth image depicting an undesirable depiction of an object;
analyzing the first image data using at least the fourth image data; and
based at least in part on analyzing the first image data, determining that the first region of the first image includes the first undesirable depiction of the object.

7. The electronic device as recited in claim 1 , further comprising:
analyzing the first region of the first image using at least one of shape analysis or color analysis; and
based at least in part on analyzing the first region of the first image, determining that the first region of the first image includes an undesirable depiction of an object,
and wherein correcting the defect in the first image or the second image is further based at least in part on the first image including the undesirable depiction of the object.

8. The electronic device as recited in claim 1 , wherein:
the first image data is generated by a camera, the first image depicting a scene that includes an object;
the second image data is generated by the camera, the second image depicting at least a portion of the scene including the object; and
the second image data is generated by the camera within a first period of time before the first image data or within a second period of time after the first image data.

9. The electronic device as recited in claim 1 , wherein the first image data includes a first resolution and the second image data includes a second resolution, the first resolution being different than the second resolution.

10. A method comprising:
receiving first image data representing a first image, the first image depicting at least a scene that includes an object;
receiving second image data representing a second image, the second image depicting at least a portion of the scene that includes the object;
determining that a first region of the first image includes a first depiction of the object;
determining that a second region of the second image includes a second depiction of the object;
determining a difference between a Discrete Cosine Transform (DCT) coefficient of the first region and a DCT coefficient of the second region;
based at least in part on the difference, assigning the first region of the first image as either a background or a foreground and assigning the second region of the second image as the other of the background or the foreground; and
based at least in part on whether the first image is assigned as the background or the foreground, generating third image data representing a third image by at least replacing the first region of the first image with the second region of the second image.

11. The method as recited in claim 10 , wherein:
determining that the first region of the first image includes the first depiction of the object comprises determining that the first region of the first image depicts at least a portion of a face, the at least the portion of the face including the object; and
determining that the second region of the second image includes the second depiction of the object comprises determining that the second region of the second image depicts at least the portion of the face.

12. The method as recited in claim 10 , further comprising:
receiving fourth image data representing a fourth image;
determining that a third region of the fourth image includes a third depiction of the object; and
based at least in part on the second depiction and the third depiction, selecting the second region of the second image for generating the third image data.

13. The method as recited in claim 10 , further comprising:
determining that the second depiction of the object includes a more desirable depiction of the object than the first depiction of the object,
and wherein generating the third image data representing the third image is based at least in part on the second depiction of the object including the more desirable depiction of the object.

14. The method as recited in claim 10 , further comprising:
determining that the first depiction of the object is associated with a sensor defect; and
determining that the second depiction of the object is not associated with the sensor defect,
and wherein generating the third image data representing the third image is based at least in part on the first depiction of the object being associated with the sensor defect and the second depiction of the object not being associated with the sensor defect.

15. The method as recited in claim 10 , wherein determining that the first region includes the first depiction of the object comprises at least:
storing fourth image data representing a fourth image, the fourth image depicting an undesirable depiction of

Assignees

• Fotonation Ltd

Inventors

• Zamfir Adrian
• Bigioi Petronel
• Corcoran Peter
• Drimbarean Alexandru
• Steinberg Eran

Classifications

• H04N5/772 Primary

  the recording apparatus and the television camera being placed in the same enclosure · CPC title

• H04N25/44

  by partially reading an SSIS array · CPC title

• H04N23/74

  by influencing the scene brightness using illuminating means · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title