Variable pre- and post-shot continuous frame buffering with automated image selection and enhancement

Patent metadata

Field	Value
Publication number	US-9807301-B1
Application number	US-201615219588-A
Country	US
Kind code	B1
Filing date	Jul 26, 2016
Priority date	Jul 26, 2016
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

A “Moment Capture System” automatically captures and buffers multiple image frames, both prior to and following digital camera shutter activation. This capture and buffering is performed in combination with real-time automatic control of camera settings (e.g., exposure time, capture rate, ISO, white balance, aperture, etc.) based on an ongoing real-time evaluation of contents and characteristics of most recent previously buffered frames. Whenever a shutter activation (e.g., a “tap”) is detected, the Moment Capture System pulls some number of pre-tap frames from the buffer and adds some number of post-tap frames to create an “image moment.” Image moments are defined as sets of sequential frames spanning a time period before and after the detected tap. In various implementations, the Moment Capture System performs automated selection of perceptually best images from the buffered frames associated with each tap. This automated selection closely emulates human selection based on subjective subtleties of human preferences.

First claim

What is claimed is:

1. A system, comprising:
a digital camera configured to continuously capture image frames to a buffer while a viewfinder of the camera is active;
applying computing functionality of the digital camera to extract a plurality of scene characteristics from one or more most recently buffered image frames as an ongoing real-time function;
applying computing functionality of the digital camera to determine and apply real-time automatic adjustments of image capture settings of the camera as an ongoing real-time function of most recently extracted scene characteristics;
the image capture settings comprising at least a determination of a first number of most recently buffered image frames to be extracted from the buffer following a detection of a shutter activation of the camera and a determination of a second number of image frames to be captured by the camera following the detection of the shutter activation;
in response to detection of the camera shutter activation, applying the computing functionality of the camera to construct and output an image moment comprising multiple image frames; and
the multiple image frames of the image moment comprising the first number of most recently buffered image frames extracted from the buffer and the second number of image frames captured by the camera following the shutter activation.

2. The system of claim 1 , the image capture settings further comprising an image frame capture rate.

3. The system of claim 1 , the image capture settings further comprising an image frame exposure setting.

4. The system of claim 1 , the image capture settings further comprising any combination of camera ISO setting, camera white balance setting, and camera aperture setting.

5. The system of claim 1 , the scene characteristics further comprising any combination of exposure, face detection, motion of subjects, and motion of the camera.

6. The system of claim 1 , further comprising applying an image selection model to automatically rank each of the image frames in the image moment in terms of a predicted subjective goodness level.

7. The system of claim 6 , further comprising automatically outputting a highest ranked one of the image frames.

8. The system of claim 1 , further comprising automatically enhancing one or more frames of the image moment via one or more image enhancement techniques to improve a subjective appearance of those one or more frames.

9. The system of claim 1 further comprising:
automatically evaluating the frames of the image moment to determine if two or more of those frames contain content that is sufficiently similar while exhibiting enough relative motion that those images would be suitable for construction of a looping animation; and
constructing a looping animation from the suitable frames of the image moment.

10. A computer-implemented process comprising applying a general-purpose computing device to perform process actions for:
causing a digital camera to continuously capture image frames to a buffer;
performing a real-time extraction of scene characteristics from a most recently buffered image frame;
determining and applying one or more real-time automatic adjustments of image capture settings of the camera as a function of the extracted scene characteristics;
the image capture settings comprising any combination of camera ISO setting, exposure time, camera white balance setting, and camera aperture setting;
the image capture settings further comprising a determination of a first number of most recently buffered image frames to be extracted from the buffer following a detection of a shutter activation of the camera and a determination of a second number of image frames to be captured by the camera following the detection of the shutter activation; and
in response to detection of the camera shutter activation, constructing an image moment comprising the first number of most recently buffered image frames extracted from the buffer and the second number of image frames captured by the camera following the shutter activation.

11. The computer-implemented process of claim 10 , the image capture settings further comprising an image frame capture rate.

12. The computer-implemented process of claim 10 , the scene characteristics further comprising any combination of exposure, face detection, motion of subjects, and motion of the camera.

13. The computer-implemented process of claim 10 , further comprising:
applying an image selection model to automatically rank each of the image frames in the image moment in terms of a predicted subjective goodness level; and
outputting one or more of the highest ranked image frames.

14. The computer-implemented process of claim 13 , further comprising automatically enhancing the one or more of the highest ranked image frames prior to outputting those frames.

15. The computer-implemented process of claim 14 , the automatic enhancement further comprising any combination of one or more image enhancement techniques to improve a subjective appearance of the one or more highest ranked image frames.

16. The computer-implemented process of claim 10 , further comprising:
automatically evaluating the frames of the image moment to determine if two or more of those frames contain content that is sufficiently similar while exhibiting enough relative motion that those images would be suitable for construction of a looping animation; and
constructing a looping animation from the suitable frames of the image moment.

17. A computer storage device having computer executable instructions stored therein, said instructions causing a computing device to execute a real-time method, comprising:
causing a digital camera to capture and buffer image frames in real-time while a viewfinder of a digital camera is active;
extracting scene characteristics from at least one most recently buffered image frame, the scene characteristics comprising any combination of exposure, face detection, motion of subjects, and motion of the camera;
determining and applying one or more real-time automatic adjustments of image capture settings of the camera as a real-time function of the extracted scene characteristics;
the image capture settings comprising a determination of a variable first number of most recently buffered image frames to be extracted from the buffer following a detection of a shutter activation of the camera and a determination of a variable second number of image frames to be captured by the camera following the detection of the shutter activation; and
in response to detection of the camera shutter activation, constructing an image moment comprising the first number of most recently buffered image frames extracted from the buffer and the second number of image frames captured by the camera following the shutter activation.

18. The computer executable instructions of claim 17 , the image capture settings further comprising any combination of camera exposure time, ISO setting, camera white balance setting, and camera aperture setting.

19. The computer executable instructions of claim 17 , further comprising:
automatically ranking each of the image frames in the image moment in terms of a predicted subjective goodness level;
automatically enhancing the one or more of the highest ranked image frames; and
outputting one or more of the enhanced image frames.

20. The computer executable instructions of claim 17 , further comprising:
automatically evaluating the frames of the image moment to determine if two or more of those frames c

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Weisberg Joshua Bryan
• Wang Baoyuan
• Kang Sing Bing

Classifications

• H04N1/2145 Primary

  of a sequence of images for selection of a single frame before final recording, e.g. from a continuous sequence captured before and after shutter-release · CPC title

• G06V10/993

  Evaluation of the quality of the acquired pattern · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/633

  for displaying additional information relating to control or operation of the camera · CPC title

• G06F18/24

  Classification techniques · CPC title