Approach for camera control

Patent metadata

Field	Value
Publication number	US-9407814-B2
Application number	US-201313782817-A
Country	US
Kind code	B2
Filing date	Mar 1, 2013
Priority date	Dec 21, 2012
Publication date	Aug 2, 2016
Grant date	Aug 2, 2016

Abstract

An approach is provided for performing back-end operations for camera control. In one example, a method includes the following: receiving a user edit via a user interface device that displays an interpretation of a scene at which a camera lens of the camera is pointing, wherein the user edit is based on user input that is associated with a selection region on the user interface device; generating an edits mask based on one or more matching image patches, which are based on the user edit and a high dynamic range (HDR) image generated by the camera; performing one or more tone mapping operations based on the edits mask and the HDR image in order to generate a tone mapped HDR image; and performing one or more metering operations based on the edits mask and the tone mapped HDR image in order to generate metering parameters for frame capturing operations.

First claim

What is claimed is:

1. A method for generating an image for display on a camera device, the method comprising:
receiving an input via a user interface device that displays an image of a scene at which a camera lens of the camera device is pointing, wherein the input is associated with a selection region of the user interface device;
generating an edits mask based on a plurality of matching image patches, which are based on the input and a high dynamic range (HDR) image generated by the camera device, wherein each matching image patch included in the plurality of matching image patches corresponds to a different image frame included in a set of sequential image frames of the scene;
performing a tone mapping operation based on the edits mask and the HDR image in order to generate a tone mapped HDR image; and
performing a metering operation based on the edits mask and the tone mapped HDR image in order to generate metering parameters for subsequent frame capturing operations.

2. The method of claim 1 , further comprising transmitting the tone mapped HDR image to the user interface device for display.

3. The method of claim 1 , wherein the metering operation comprises:
quantifying metering requirements based on the edits mask and the tone mapped HDR image;
metering the tone mapped HDR image to calculate metering parameters; and
providing to the camera device the metering parameters that affect the subsequent frame capturing operations for generating the HDR image.

4. The method of claim 1 , wherein each matching image patch included in the plurality of matching image patches includes a subset of pixels included in the selection region of the corresponding image frame.

5. The method of claim 1 , wherein generating the edits mask comprises:
processing a sub-sampled set of image patches to generate an intermediate image mask; and
performing an edge-aware up-sampling operation on the intermediate mask based on a domain transform with respect to an edge of an object in image frames captured via the camera lens.

6. The method of claim 5 , wherein the edge-aware up-sampling operation provides data to a processor that, in response, performs operations at a higher rate relative to operations prior to the edge-aware up-sampling operation.

7. The method of claim 5 , wherein the edge-aware up-sampling operation provides data to a processor that enables the processor to distinguish between a texture associated with the object and a texture associated with another part of the scene.

8. The method of claim 1 , wherein the high dynamic range image is generated by blending image frames of a sliding window, wherein the sliding window includes, at a first time, a first sequence of two or more adjacent image frames captured via the camera lens.

9. The method of claim 8 , wherein the sliding window includes, at a second time, a second sequence of two or more adjacent image frames captured via the camera lens.

10. The method of claim 1 , wherein the scene is a live scene that includes objects that are moving.

11. A non-transitory computer-readable storage medium including instructions that, when executed by a processor, cause the processor to generate an image for display on a camera device, by performing the steps of:
receiving an input via a user interface device that displays an image of a scene at which a camera lens of the camera device is pointing, wherein the input is associated with a selection region of the user interface device;
generating an edits mask based on a plurality of matching image patches, which are based on the input and a high dynamic range (HDR) image generated by the camera device, wherein each matching image patch included in the plurality of matching image patches corresponds to a different image frame included in a set of sequential image frames of the scene;
performing a tone mapping operation based on the edits mask and the HDR image in order to generate a tone mapped HDR image; and
performing a metering operation based on the edits mask and the tone mapped HDR image in order to generate metering parameters for subsequent frame capturing operations.

12. The non-transitory computer-readable storage medium of claim 11 , further comprising transmitting the tone mapped HDR image to the user interface device for display.

13. The non-transitory computer-readable storage medium of claim 11 , wherein the metering operation comprises:
quantifying metering requirements based on the edits mask and the tone mapped HDR image;
metering the tone mapped HDR image to calculate metering parameters; and
providing to the camera device the metering parameters that affect the subsequent frame capturing operations for generating the HDR image.

14. The non-transitory computer-readable storage medium of claim 11 , wherein each matching image patch included in the plurality of matching image patches includes a subset of pixels included in the selection region of the corresponding image frame.

15. The non-transitory computer-readable storage medium of claim 11 , wherein generating the edits mask comprises:
processing a sub-sampled set of image patches to generate an intermediate image mask; and
performing an edge-aware up-sampling operation on the intermediate mask based on a domain transform with respect to an edge of an object in image frames captured via the camera lens.

16. The non-transitory computer-readable storage medium of claim 15 , wherein the edge-aware up-sampling operation provides data to a processor that, in response, performs operations at a higher rate relative to operations prior to the edge-aware up-sampling operation.

17. The non-transitory computer-readable storage medium of claim 15 , wherein the edge-aware up-sampling operation provides data to a processor that enables the processor to distinguish between a texture associated with the object and a texture associated with another part of the scene.

18. The non-transitory computer-readable storage medium of claim 11 , wherein the high dynamic range image is generated by blending image frames of a sliding window, wherein the sliding window includes, at a first time, a first sequence of two or more adjacent image frames captured via the camera lens.

19. The non-transitory computer-readable storage medium of claim 18 , wherein the sliding window includes, at a second time, a second sequence of two or more adjacent image frames captured via the camera lens.

20. A camera control system, comprising:
a mask generator device configured to:
receive an input via a user interface device that displays an image of a scene at which a camera lens of a camera device is pointing, wherein the input is associated with a selection region of the user interface device, and
generate an edits mask based on a plurality of matching image patches, which are based on the input and a high dynamic range (HDR) image generated by the camera device, wherein each matching image patch included in the plurality of matching image patches corresponds to a different image frame included in a set of sequential image frames of the scene; and

a metering device configured to:
perform a tone mapping operation based on the edits mask and the HDR image in order to generate a tone mapped HDR image, and
perform a metering operation based on the edits mask and the tone mapped HDR image in order to generate metering parameters for subsequent frame capturing operations.

21. The method of claim MN, further comprising increasing a quantity of image frames included in the s

Assignees

• Nvidia Corp

Inventors

• Pajak Dawid Stanislaw
• Baek Jongmin
• Pulli Kari

Classifications

• H04N23/635

  Region indicators; Field of view indicators · CPC title

• H04N23/62 Primary

  Control of parameters via user interfaces · CPC title

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title

• H04N5/2621 Primary

  Cameras specially adapted for the electronic generation of special effects during image pickup, e.g. digital cameras, camcorders, video cameras having integrated special effects capability · CPC title

• H04N5/23293

  Electricity · mapped topic