Systems and methods for autonomously generating photo summaries

Patent metadata

Field	Value
Publication number	US-2016119536-A1
Application number	US-201414526355-A
Country	US
Kind code	A1
Filing date	Oct 28, 2014
Priority date	Oct 28, 2014
Publication date	Apr 28, 2016
Grant date	—

Abstract

Implementations of the disclosed technology include techniques for autonomously collecting image data, and generating photo summaries based thereon. In some implementations, a plurality of images may be autonomously sampled from an available stream of image data. For example, a camera application of a smartphone or other mobile computing device may present a live preview based on a stream of data from an image capture device. The live stream of image capture data may be sampled and the most interesting photos preserved for further filtering and presentation. The preserved photos may be further winnowed as a photo session continues and an image object generated summarizing the remaining photos. Accordingly, image capture data may be autonomously collected, filtered, and formatted to enable a photographer to see what moments they missed manually capturing during a photo session.

First claim

We claim:

1 . A method comprising:
responsive to activation of an image-capture application, receiving, by a computing device, from an image capture device, a live stream of image data; storing, autonomously by the computing device, in a first buffer, while receiving the live stream of image data, a set of images based on the live stream of image data; identifying, by the computing device, one or more predetermined features among a subset of the set of images; generating, by the computing device, based on the subset of the set of images, an image object representing a summary of the live stream of image data; and outputting, by the computing device, for display at a display device, an indication of the image object.

2 . The method of claim 1 , wherein the one or more predetermined features among the subset of images includes at least one of sharpness, presence of faces, facial expressions, illumination quality, image saliency, motion saliency, and color diversity.

3 . The method of claim 1 , wherein the subset of images is stored in a second buffer, and responsive to a number or size of images in the subset of images reaching a predetermined threshold, determining, based on predetermined criteria, a least important image in the subset of images to the summary of the stream of image data, and removing the least important image from the subset of images to provide room for a new image to be added to the subset of images.

4 . The method of claim 1 , wherein:
storing the plurality of images in the first buffer comprises storing, at a first resolution, a first image of the plurality of images; and the determining the indication of one or more predetermined features among the subset of images is based on analyzing a version of the first image at a second resolution less than the first resolution.

5 . The method of claim 1 , wherein the autonomously storing is automatically initiated responsive to activating an image-capture application.

6 . The method of claim 1 , wherein the first buffer is a circular buffer, and the stream of image data exceeds a size of the circular buffer.

7 . The method of claim 8 , wherein the size of the circular buffer corresponds to approximately three seconds or less of the stream of image data.

8 . A computer program product comprising a non-transitory computer readable medium that stores instructions that, when executed by a computing device, cause the computing device to perform a method comprising:
receiving, by the computing device, from an image capture device, a stream of image data; outputting, by the computing device, for display at a display device, a live preview based on the stream of image data; storing, autonomously by the computing device, in a first buffer, during the outputting of the live preview, a set of images from the stream of image data; identifying, by the computing device, one or more predetermined features among a subset of the set of images; generating, by the computing device, based on the subset of images, an image object representing a summary of the stream of image data; and outputting, by the computing device, for display at the display device, an indication of the image object.

9 . The computer program product of claim 10 , wherein the outputting of the indication of the image object is responsive to stopping the outputting of the live preview.

10 . The computer program product of claim 10 , wherein the outputting of the indication of the image object is responsive to receiving user input corresponding to an image capture command.

11 . The computer program product of claim 10 , wherein the autonomously storing is initiated responsive to receiving first user input corresponding to an image-capture command.

12 . The computer program product of claim 13 , wherein the autonomously storing is concluded responsive to receiving second user input corresponding to an image-capture command.

13 . A system comprising:
a computing device; an image capture device in communication with the computing device; at least one memory operatively coupled to the computing device and configured for storing data and instructions that, when executed by the computing device, cause the computing device to perform a method comprising:
receiving, by a computing device, from an image capture device, a stream of image data;
outputting, by the computing device, for display at a display device, a live preview based on the stream of image data;
sampling, automatically by the computing device, during the outputting of the live preview, a set of images from the stream of image data, and storing the set of images in a first buffer;
identifying, by the computing device, one or more predetermined features among a subset of the set of images;
generating, by the computing device, based on the subset of images, an image object representing a summary of the stream of image data; and
outputting, by the computing device, for display at the display device, an indication of the image object.

14 . The method of claim 13 , wherein a rate of the sampling is based on a running measure of motion associated with a plurality of images previously sampled from the image stream.

15 . The method of claim 14 , wherein the rate of the sampling is proportional to an integral of the running measure of motion.

16 . The system of claim 15 , wherein the image object is at least partially embodied in an animation.

17 . The system of claim 16 , wherein the generating the image object is based on determining the subset of images exceeds at least one of a predetermined blurriness threshold and predetermined specific motion threshold.

18 . The system of claim 15 , wherein the image object is at least partially embodied in a concurrent display of a plurality of still images.

19 . The system of claim 18 , wherein the generating the image object is based on determining the subset of images exceeds at least one of a predetermined sharpness threshold and a predetermined general motion threshold.

20 . The system of claim 15 , wherein the image object comprises a single image.

Assignees

• Google Inc

Inventors

• Carceroni Rodrigo Lima
• Renn Marius
• Newberger Alan
• Häberling Sascha
• Mintz Jacob
• Huibers Andrew

Classifications

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/611 Primary

  where the recognised objects include parts of the human body · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• G06V20/47 Primary

  Detecting features for summarising video content · CPC title

• H04N5/23222 Primary

  Electricity · mapped topic