Closed loop photography

Patent metadata

Field	Value
Publication number	US-12477198-B2
Application number	US-202318334585-A
Country	US
Kind code	B2
Filing date	Jun 14, 2023
Priority date	Jun 14, 2022
Publication date	Nov 18, 2025
Grant date	Nov 18, 2025

Abstract

Improved techniques of generating photographs using a camera or other image capture device includes the use of “soft” indicators such as haptics, audio, heads-up display, and other modalities to warn the user of a potential occlusion before a user activates a shutter trigger.

First claim

What is claimed is:

1 . An apparatus, comprising:
a camera including:
a shutter trigger configured to cause a shutter to open; and
an occlusion device configured to determine whether an object will occlude a scene while the shutter is open, the occlusion device including:
a proximity device configured to determine whether the object is moving toward the shutter trigger based on a temporal pattern of radiation reflected from the object; and
a notification device configured to, in response to the proximity device determining that the object is moving toward the shutter trigger, transmit a notification that the object is occluding the scene.

2 . The apparatus as in claim 1 , wherein the proximity device includes:
a source of electromagnetic radiation configured to transmit electromagnetic radiation propagating away from the camera; and processing circuitry configured to determine whether the object is moving to cause the shutter to be opened based on the temporal pattern of the electromagnetic radiation reflected from the object.

3 . The apparatus as in claim 2 , wherein the processing circuitry configured to determine whether the object is moving to cause the shutter to be opened is further configured to:
perform a classification operation on the temporal pattern of electromagnetic radiation reflected toward the camera to produce a classification of whether or not the object is moving to cause the shutter to be opened, the classification operation being based on a support vector machine trained to perform the classification operation based on a proximity of the object to the camera and an amplitude of the temporal pattern of electromagnetic radiation reflected toward the camera.

4 . The apparatus as in claim 3 , wherein the classification of whether or not the object is moving to cause the shutter to be opened is based on whether a classification score is less than or greater than a threshold, the classification score including a linear combination of a proximity of the object to the camera and an amplitude of the temporal pattern of electromagnetic radiation propagating toward the camera, coefficients of the proximity and the amplitude being based on a device upon which the camera is disposed.

5 . The apparatus as in claim 2 , wherein the processing circuitry is first processing circuitry, and
wherein the proximity device further includes second processing circuitry configured to:
determine, in response to the first processing circuitry determining that the object is moving to cause the shutter to be opened, a degree to which the object is occluding the scene,
wherein the notification is transmitted in response to the degree being larger than a threshold.

6 . The apparatus as in claim 5 , wherein the second processing circuitry is further configured to, in response to the first processing circuitry determining that the object is moving to cause the shutter to be opened:
open and close the shutter a specified number of times at a specified rate to produce a plurality of image frames; and wherein the second processing circuitry configured to determine the degree to which the object is occluding the scene is further configured to:
input the plurality of image frames into a convolutional neural network configured to produce an occlusion strength indicating the degree to which the object is occluding the scene.

7 . The apparatus as in claim 6 , wherein the second processing circuitry configured to input the plurality of image frames into the convolutional neural network is further configured to:
generate an optical flow measure based on the plurality of image frames, the optical flow measure indicating a measure of consistency of occlusion between each pair of sequential frames of the plurality of image frames; and input the optical flow measure into the convolutional neural network.

8 . The apparatus as in claim 6 , wherein the second processing circuitry configured to transmit notification data indicating that the scene is occluded by the object is further configured to:
transmit an audio signal having a volume based on the occlusion strength.

9 . The apparatus as in claim 5 , wherein the camera is disposed on a frame of a smartglasses system, and
wherein the second processing circuitry configured to transmit notification data indicating that the scene is occluded by the object is further configured to:
transmit a video signal configured to be displayed on a display of the smartglasses system.

10 . The apparatus as in claim 2 , wherein the source of electromagnetic radiation is disposed within the shutter trigger.

11 . A method, comprising:
receiving electromagnetic radiation reflected by an object; determining whether the object is occluding a scene based on a temporal pattern of the electromagnetic radiation reflected by the object; and in response to determining that the object is occluding the scene, transmitting a notification that the object is occluding the scene.

12 . The method as in claim 11 , wherein determining whether the object is occluding the scene includes:
transmitting electromagnetic radiation propagating away from a camera including a shutter trigger configured to cause a shutter to open; and determining whether the object is moving to cause the shutter to be opened based on the temporal pattern of the electromagnetic radiation reflected from the object.

13 . The method as in claim 12 , wherein determining whether the object is moving to cause the shutter to be opened includes:
performing a classification operation on the temporal pattern of electromagnetic radiation reflected toward the camera to produce a classification of whether or not the object is moving to cause the shutter to be opened, the classification operation being based on a support vector machine trained to perform the classification operation based on a proximity of the object to the camera and an amplitude of the temporal pattern of electromagnetic radiation reflected toward the camera.

14 . The method as in claim 13 , wherein the classification of whether or not the object is moving to cause the shutter to be opened is based on whether a classification score is less than or greater than a threshold, the classification score including a linear combination of a proximity of the object to the camera and an amplitude of the temporal pattern of electromagnetic radiation propagating toward the camera, coefficients of the proximity and the amplitude being based on a device upon which the camera is disposed.

15 . The method as in claim 12 , further comprising:
determining, in response to determining that the object is moving to cause the shutter to be opened, a degree to which the object is occluding the scene, wherein the notification is transmitted in response to the degree being larger than a threshold.

16 . The method as in claim 15 , further comprising, in response to determining that the object is moving to cause the shutter to be opened:
opening and closing the shutter a specified number of times at a specified rate to produce a plurality of image frames; and wherein determining the degree to which the object is occluding the scene includes:
inputting the plurality of image frames into a convolutional neural network configured to produce an occlusion strength indicating the degree to which the object is occluding the scene.

17 . The method as in claim 16 , wherein inputting the plurality of image frames into the convolutional neural network includes:
generating an optical flow measure based on the plurality of

Assignees

• Google Llc

Inventors

• Shin Dongeek

Classifications

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G03B9/58 Primary

  Means for varying duration of "open" period of shutter · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06T7/0016

  involving temporal comparison · CPC title