Method and apparatus for finding and using video portions that are relevant to adjacent still images

Patent metadata

Field	Value
Publication number	US-10170157-B2
Application number	US-201514865752-A
Country	US
Kind code	B2
Filing date	Sep 25, 2015
Priority date	Jun 7, 2015
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

The invention relates to systems, methods, and computer readable media for responding to a user snapshot request by capturing anticipatory pre-snapshot image data as well as post-snapshot image data. The captured information may be used, depending upon the embodiment, to create archival image information and image presentation information that is both useful and pleasing to a user. The captured information may automatically be trimmed or edited to facilitate creating an enhanced image, such as a moving still image. Varying embodiments of the invention offer techniques for trimming and editing based upon the following: exposure, brightness, focus, white balance, detected motion of the camera, substantive image analysis, detected sound, image metadata, and/or any combination of the foregoing.

First claim

We claim:

1. A method of producing an enhanced image comprising:
capturing a first plurality of images, wherein the first plurality of images comprises:
a snapshot image, wherein the snapshot image is captured in response to receiving a snapshot indication through a user interface;
a pre-capture image sequence, wherein the pre-capture image sequence comprises a plurality of images captured prior to capturing the snapshot image, and wherein the snapshot indication is received after capturing at least one of the plurality of images of the pre-capture image sequence; and
a post-capture image sequence, wherein the post-capture image sequence comprises a plurality of images captured after capturing the snapshot image;

obtaining motion information for each of the first plurality of images;
determining, for each image of the pre-capture image sequence and the post-capture image sequence, whether the motion information of a respective image meets a first motion criterion, wherein the first motion criterion comprises a comparison between the motion information of the respective image and the motion information of the snapshot image;
identifying a first subset of images from among the pre-capture image sequence that meet the first motion criterion;
identifying a second subset of images from among the post-capture image sequence that meet the first motion criterion; and
creating an enhanced image comprising the snapshot image, the first subset of images, and the second subset of images.

2. The method of claim 1 , wherein the first subset of images comprises a consecutive set of images, beginning with an image in the pre-capture image sequence and ending with the image captured immediately prior to the snapshot image.

3. The method of claim 1 , wherein the second subset of images comprises a consecutive set of images, beginning with the image captured immediately after the snapshot image and ending with an image in the post-capture image sequence.

4. The method of claim 1 , wherein the motion information for at least one of the first plurality of images comprises information obtained from an accelerometer or information obtained from a gyroscope.

5. The method of claim 1 , wherein the motion information for at least one of the first plurality of images comprises information associated with a direction of motion for the at least one of the first plurality of images.

6. The method of claim 1 , wherein the first motion criterion further comprises determining whether a direction of the motion information for an image in the first plurality of images is consistent with a direction of the motion information for the snapshot image.

7. The method of claim 1 , wherein the first motion criterion further comprises determining whether a direction of the motion information for an image in the first plurality of images is consistent with a direction of the motion information for one or more other images in the first plurality of images.

8. The method of claim 1 , wherein the first motion criterion further comprises a difference between the motion information of the respective image and the motion information of the snapshot image being less than a predetermined threshold amount.

9. A non-transitory computer readable medium having stored thereon a program, the program comprising one or more instructions that, when executed by a processor, cause the processor to:
capture a first plurality of images, wherein the first plurality of images comprises:
a snapshot image, wherein the snapshot image is captured in response to receiving a snapshot indication through a user interface;
a pre-capture image sequence, wherein the pre-capture image sequence comprises a plurality of images captured prior to capturing the snapshot image, and wherein the snapshot indication is received after capturing at least one of the plurality of images of the pre-capture image sequence; and
a post-capture image sequence, wherein the post-capture image sequence comprises a plurality of images captured after capturing the snapshot image;

obtain motion information for each of the first plurality of images;
determine, for each image of the pre-capture image sequence and the post-capture image sequence, whether the motion information of a respective image meets a first motion criterion, wherein the first motion criterion comprises a comparison between the motion information of the respective image and the motion information of the snapshot image;
identify a first subset of images from among the pre-capture image sequence that meet the first motion criterion;
identify a second subset of images from among the post-capture image sequence that meet the first motion criterion; and
create an enhanced image comprising the snapshot image, the first subset of images, and the second subset of images.

10. The non-transitory computer readable medium of claim 9 , wherein the first subset of images comprises a consecutive set of images, beginning with an image in the pre-capture image sequence and ending with the image captured immediately prior to the snapshot image.

11. The non-transitory computer readable medium of claim 9 , wherein the second subset of images comprises a consecutive set of images, beginning with the image captured immediately after the snapshot image and ending with an image in the post-capture image sequence.

12. The non-transitory computer readable medium of claim 9 , wherein the motion information for at least one of the first plurality of images comprises information obtained from an accelerometer or information obtained from a gyroscope.

13. The non-transitory computer readable medium of claim 9 , wherein the motion information for at least one of the first plurality of images comprises information associated with a direction of motion for the at least one of the first plurality of images.

14. The non-transitory computer readable medium of claim 9 , wherein the first motion criterion further comprises a difference between the motion information of the respective image and the motion information of the snapshot image being less than a predetermined threshold amount.

15. A system comprising:
one or more CPUs;
one or more cameras for capturing images; and
a memory for storing program instructions for the one or more CPUs, wherein the instructions, when executed, cause the one or more CPUs to:
capture a first plurality of images, wherein the first plurality of images comprises:
a snapshot image, wherein the snapshot image is captured in response to receiving a snapshot indication through a user interface;
a pre-capture image sequence, wherein the pre-capture image sequence comprises a plurality of images captured prior to capturing the snapshot image, and wherein the snapshot indication is received after capturing at least one of the plurality of images of the pre-capture image sequence; and
a post-capture image sequence, wherein the post-capture image sequence comprises a plurality of images captured after capturing the snapshot image;

obtain motion information for each of the first plurality of images;
determine, for each image of the pre-capture image sequence and the post-capture image sequence, whether the motion information of a respective image meets a first motion criterion, wherein the first motion criterion comprises a comparison between the motion information of the respective image and the motion information of the snapshot image;
identify a first subset of images from among the pre-capture image sequence that meet the first motion criterion;
identify a second subset of images from among the post-capture image sequence that meet t

Assignees

• Apple Inc

Inventors

• Molgaard Claus
• Keating Brett M
• Williams George E
• Zuliani Marco
• Wong Vincent Y
• Doepke Frank
• Tira-Thompson Ethan J

Classifications

• H04N5/2621 Primary

  Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability · CPC title

• G11B27/34 Primary

  Indicating arrangements  {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)} · CPC title

• H04N23/73 Primary

  by influencing the exposure time · CPC title

• H04N23/667

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title

• H04N23/741

  by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors · CPC title