Self-portrait enhancement techniques

Patent metadata

Field	Value
Publication number	US-10121229-B2
Application number	US-201514984805-A
Country	US
Kind code	B2
Filing date	Dec 30, 2015
Priority date	Aug 30, 2013
Publication date	Nov 6, 2018
Grant date	Nov 6, 2018

Abstract

Systems and approaches are provided for optimizing self-portraiture. The background of the self-portrait can be enhanced by image registration or stitching techniques of images captured using one or more conventional cameras. Multiple standard resolution images can be stitched together to generate a panoramic or a composite image of a higher resolution. Foreground elements, such as one or more representations of users, can also be enhanced in various ways. The representations of the users can be composited to exclude undesirable elements, such as image data of one of the users extending her arm to capture the self-portrait. An ideal pose of the users can automatically be selected and other image enhancements, such as histogram optimization, brightness and contrast optimization, color-cast correction, or reduction or removal of noise, can automatically be performed to minimize user effort in capturing self-portraits.

First claim

What is claimed is:

1. A computer-implemented method, comprising:
obtaining a plurality of images;
determining one or more background elements from each of the plurality of images;
determining a plurality of foreground elements from each of the plurality of images;
generating a background image comprising a first composite of at least a portion of the one or more background elements;
determining a score of each of the plurality of foreground elements according to one or more visual characteristics;
obtaining a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and
generating a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements.

2. The computer-implemented method of claim 1 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.

3. The computer-implemented method of claim 1 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.

4. The computer-implemented method of claim 1 , further comprising:
obtaining one or more selections relating to display of the background image,
wherein the self-portrait image is rendered based at least in part upon the one or more selections.

5. The computer-implemented method of claim 4 , wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image.

6. A computing device, comprising:
one or more processors;
one or more cameras;
memory including instructions that, when executed by the one or more processors, cause the computing device to:
obtain a plurality of images;
determine one or more background elements from each of the plurality of images;
determine a plurality of foreground elements from each of the plurality of images;
generate a background image comprising a first composite of at least a portion of the one or more background elements;
determine a score of each of the plurality of foreground elements according to one or more visual characteristics;
obtaining a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and
generate a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements.

7. The computing device of claim 6 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.

8. The computing device of claim 6 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.

9. The computing device of claim 6 , wherein the instructions, when executed to generate the background image, further enable the computing device to:
generate a preliminary background image by stitching together at least the portion of the one or more background elements; and
fill in at least a portion of the preliminary background image using an inpainting process.

10. The computing device of claim 6 , wherein the instructions, when executed by the one or more processors, further enable the computing device to:
obtain one or more selections relating to display of the background image,
wherein the self-portrait image is rendered based at least in part upon the one or more selections.

11. The computing device of claim 10 , wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image.

12. A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to:
obtain a plurality of images;
determine one or more background elements from each of the plurality of images;
determine a plurality of foreground elements from each of the plurality of images;
generate a background image comprising a first composite of at least a portion of the one or more background elements;
determine a score of each of the plurality of foreground elements according to one or more visual characteristics;
obtain a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and
generate a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.

15. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions, when executed to generate the background image, further cause the processor to:
generate a preliminary background image by stitching together at least the portion of the one or more background elements; and
fill in at least a portion of the preliminary background image using an inpainting process.

16. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions, when executed by the processor, further cause the processor to:
obtain one or more selections relating to display of the background image, wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image; and
wherein the self-portrait image is rendered based at least in part upon the one or more selections.

Assignees

• A9 Com Inc

Inventors

• Gray Douglas Ryan
• Taylor Colin Jon
• Lin Xiaofan

Classifications

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N5/272 Primary

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title

• H04N5/23238

  Electricity · mapped topic

• G06T2207/30201

  Face · CPC title