Self-portrait enhancement techniques

Patent metadata

Field	Value
Publication number	US-2016110848-A1
Application number	US-201514984805-A
Country	US
Kind code	A1
Filing date	Dec 30, 2015
Priority date	Aug 30, 2013
Publication date	Apr 21, 2016
Grant date	—

Abstract

Systems and approaches are provided for optimizing self-portraiture. The background of the self-portrait can be enhanced by image registration or stitching techniques of images captured using one or more conventional cameras. Multiple standard resolution images can be stitched together to generate a panoramic or a composite image of a higher resolution. Foreground elements, such as one or more representations of users, can also be enhanced in various ways. The representations of the users can be composited to exclude undesirable elements, such as image data of one of the users extending her arm to capture the self-portrait. An ideal pose of the users can automatically be selected and other image enhancements, such as histogram optimization, brightness and contrast optimization, color-cast correction, or reduction or removal of noise, can automatically be performed to minimize user effort in capturing self-portraits.

First claim

What is claimed is:

1 . A computer-implemented method, comprising:
obtaining a plurality of images; determining one or more background elements from the plurality of images; determining a plurality of foreground elements from the plurality of images; generating a background image comprising a first composite of at least a portion of the one or more background elements; ranking the plurality of foreground elements according to one or more visual characteristics; and generating a self-portrait image, the self-portrait image comprising at least one of the plurality of foreground elements based on the ranking of the plurality of foreground elements and the background image.

2 . The computer-implemented method of claim 1 , further comprising:
displaying the plurality of foreground elements as a ranked list; and receiving a selection of the at least one of the plurality of foreground elements.

3 . The computer-implemented method of claim 1 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.

4 . The computer-implemented method of claim 1 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.

5 . The computer-implemented method of claim 1 , wherein generating the background image includes:
generating a preliminary background image by stitching together at least the portion of the one or more background elements; and filling in at least a portion of the preliminary background image using an inpainting process.

6 . The computer-implemented method of claim 1 , further comprising:
obtaining one or more selections relating to display of the background image, wherein the self-portrait image is rendered based at least in part upon the one or more selections.

7 . The computer-implemented method of claim 6 , wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image.

8 . A computing device, comprising:
one or more processors; one or more cameras; memory including instructions that, when executed by the one or more processors, cause the computing device to:
obtain a plurality of images;
determine one or more background elements from the plurality of images;
determine a plurality of foreground elements from the plurality of images;
generate a background image comprising a first composite of at least a portion of the one or more background elements;
rank the plurality of foreground elements according to one or more visual characteristics; and
generate a self-portrait image, the self-portrait image comprising at least one of the plurality of foreground elements based on the ranking of the plurality of foreground elements and the background image.

9 . The computing device of claim 8 , wherein the instructions, when executed by the one or more processors, further enable the computing device to:
display the plurality of foreground elements as a ranked list; and receive a selection of the at least one of the plurality of foreground elements.

10 . The computing device of claim 8 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.

11 . The computing device of claim 8 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.

12 . The computing device of claim 8 , wherein the instructions, when executed to generate the background image, further enable the computing device to:
generate a preliminary background image by stitching together at least the portion of the one or more background elements; and fill in at least a portion of the preliminary background image using an inpainting process.

13 . The computing device of claim 8 , wherein the instructions, when executed by the one or more processors, further enable the computing device to:
obtain one or more selections relating to display of the background image, wherein the self-portrait image is rendered based at least in part upon the one or more selections.

14 . The computing device of claim 13 , wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image.

15 . A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to:
obtain a plurality of images; determine one or more background elements from the plurality of images; determine a plurality of foreground elements from the plurality of images; generate a background image comprising a first composite of at least a portion of the one or more background elements; rank the plurality of foreground elements according to one or more visual characteristics; and generate a self-portrait image, the self-portrait image comprising at least one of the plurality of foreground elements based on the ranking of the plurality of foreground elements and the background image.

16 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the processor, further cause the processor to:
display the plurality of foreground elements as a ranked list; and receive a selection of the at least one of the plurality of foreground elements.

17 . The non-transitory computer-readable storage medium of claim 15 , wherein the plurality of foreground elements include at least a portion of representations of one or more users.

18 . The non-transitory computer-readable storage medium of claim 15 , wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users.

19 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed to generate the background image, further cause the processor to:
generate a preliminary background image by stitching together at least the portion of the one or more background elements; and fill in at least a portion of the preliminary background image using an inpainting process.

20 . The non-transitory computer-readable storage medium of claim 15 , wherein the instructions, when executed by the processor, further cause the processor to:
obtain one or more selections relating to display of the background image, wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image; and wherein the self-portrait image is rendered based at least in part upon the one or more selections.

Assignees

• A9 Com Inc

Inventors

• Gray Douglas Ryan
• Taylor Colin Jon
• Lin Xiaofan

Classifications

• H04N5/272 Primary

  Means for inserting a foreground image in a background image, i.e. inlay, outlay · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T2207/30201

  Face · CPC title