Generating a composite image from a physical item

Patent metadata

Field	Value
Publication number	US-10257485-B2
Application number	US-201615350840-A
Country	US
Kind code	B2
Filing date	Nov 14, 2016
Priority date	Jun 8, 2016
Publication date	Apr 9, 2019
Grant date	Apr 9, 2019

Abstract

A computer-implemented method includes capturing, with a camera, a first image of a physical item at a first camera position, detecting borders associated with the physical item, based on the first image, generating an overlay that includes a plurality of objects that are positioned within one or more of the borders associated with the physical item, capturing, with the camera, subsequent images of the physical item, where each subsequent image is captured with a respective subsequent camera position, and during capture of the subsequent images, displaying an image preview that includes the overlay. The method further includes establishing correspondence between pixels of the first image and pixels of each of the subsequent images and generating a composite image of the physical item, where each pixel value of the composite image is based on corresponding pixel values of the first image and the subsequent images.

First claim

What is claimed is:

1. A method comprising:
capturing, with a camera, a first image of a physical photograph at a first camera position;
detecting borders associated with edges of the physical photograph;
based on the first image, generating an overlay that includes a plurality of objects that are positioned within one or more of the borders associated with the physical photograph;
capturing, with the camera, subsequent images of the physical photograph, wherein each subsequent image is captured with a respective subsequent camera position;
during capture of the subsequent images, displaying an image preview that includes the overlay, wherein:
the plurality of objects remains fixed in the overlay as the camera is moved;
in response to a predetermined number of captures of subsequent images at each respective subsequent camera position, an attribute of an object of the plurality of objects is updated; and
updating the attribute of the plurality of objects corresponds to a completion of capture;

establishing correspondence between pixels of the first image and pixels of each of the subsequent images; and
generating a composite image of the physical photograph, wherein each pixel value of the composite image is based on corresponding pixel values of the first image and the subsequent images and the composite image depicts reduced or eliminated glare that occurs when capturing one or more of the first image and the subsequent images.

2. The method of claim 1 , wherein:
the overlay further comprises a reticle,
the reticle moves responsive to moving the camera; and
the subsequent images are captured responsive to at least a portion of the reticle overlapping with the object.

3. The method of claim 1 , wherein:
responsive to a size of the first image exceeding a predetermined threshold value, dividing the overlay into sections and positioning each of the plurality of objects in a center of a respective section.

4. The method of claim 1 , wherein:
responsive to a confidence value associated with detection of the first image exceeding a threshold confidence value, dividing the overlay into sections and positioning each of the plurality of objects in a center of a respective section.

5. The method of claim 1 , wherein:
the attribute is color; and
updating the color of the plurality of objects causes an indication of completion to be displayed.

6. The method of claim 1 , wherein:
responsive to updating the attribute of the object of the plurality of objects, the object is removed from the overlay.

7. The method of claim 1 , wherein each pixel value of the composite image is a minimum value of the corresponding pixel values of the first image and the subsequent images.

8. The method of claim 1 , wherein:
generating the composite image further includes for each pixel value of the composite image, selecting a tentative pixel value and comparing the tentative pixel value to surrounding pixel values for pixels that surround a pixel corresponding to the pixel value to confirm that each of the surrounding pixel values is within a threshold difference of the tentative pixel value.

9. The method of claim 8 , wherein:
responsive to a difference between the tentative pixel value and one of the surrounding pixel values exceeding the threshold difference, modifying the tentative pixel value to be within the threshold difference.

10. A non-transitory computer readable medium with instructions that, when executed by one or more computers, cause the one or more computers to perform operations, the operations comprising:
capturing a first image of a physical photograph with a camera;
detecting borders associated with edges of the physical photograph;
based on the first image, generating an overlay that includes a reticle and a plurality of objects that are positioned within one or more of the borders associated with the physical photograph;
capturing subsequent images of the physical photograph, wherein each subsequent image is captured with a respective subsequent camera position;
during capture of the subsequent images, providing the first image with the overlay, wherein:
the plurality of objects remain fixed in the overlay as the camera is moved,
the reticle moves responsive to moving the camera,
the subsequent images are captured responsive to at least a portion of the reticle overlapping with an object of the plurality of objects,
in response to a predetermined number of captures of subsequent images at each respective subsequent camera position, the object is updated, and
a completion of capture occurs responsive to moving the camera until the plurality of objects are updated;

establishing correspondence between pixels of the first image and pixels of each of the subsequent images; and
generating a composite image of the physical photograph, wherein each pixel value of the composite image is based on corresponding pixel values of the first image and the subsequent images.

11. The computer-readable medium of claim 10 , wherein:
updating the object includes updating a color of the object; and
updating the color of the plurality of objects causes an indication of completion to be displayed.

12. The computer-readable medium of claim 10 , wherein the overlay further includes an arrow that points to a direction to move the reticle.

13. The computer-readable medium of claim 10 , wherein each pixel value of the composite image is a minimum value of the corresponding pixel values of the first image and the subsequent images.

14. The computer-readable medium of claim 10 , wherein:
generating the composite image further includes for each pixel value of the composite image, selecting a tentative pixel value and comparing the tentative pixel value to surrounding pixel values for pixels that surround a pixel corresponding to the pixel value to confirm that each of the surrounding pixel values are within a threshold difference of the tentative pixel value.

15. The computer-readable medium of claim 14 , wherein:
responsive to a difference between the tentative pixel value and one of the surrounding pixel values exceeding the threshold difference, modifying the tentative pixel value such that a difference between the modified tentative pixel value and the one of the surrounding pixel values does not exceed the threshold difference.

16. A system comprising:
one or more processors; and
a memory that stores instructions executable by the one or more processors that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
receiving, from a camera, a first image of a physical photograph at a first camera position;
detecting borders associated with edges of the physical photograph;
based on the first image, generating an overlay that includes a plurality of objects that are positioned within one or more of the borders associated with the physical photograph;
receiving, from the camera, subsequent images of the physical photograph, wherein each subsequent image is captured with a respective subsequent camera position;
during capture of the subsequent images, displaying an image preview that includes the overlay, wherein:
the plurality of objects remains fixed in the overlay as the camera is moved;
in response to a predetermined number of captures of subsequent images at each respective subsequent camera position, an attribute of an object of the plurality of objects is updated; and
updating the attribute of the plurality of objects corresponds to a completion of capture;

establishing correspondence between pixels

Assignees

• Google Inc
• Google Llc

Inventors

• Winn Julia
• Lieb David
• Henry Eric

Classifications

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/743

  Bracketing, i.e. taking a series of images with varying exposure conditions · CPC title

• H04N23/12

  with one sensor only · CPC title

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N1/00183 Primary

  Photography assistance, e.g. displaying suggestions to the user · CPC title