Artificial aperture adjustment for synthetic depth of field rendering

Patent metadata

Field	Value
Publication number	US-11682108-B2
Application number	US-202117473531-A
Country	US
Kind code	B2
Filing date	Sep 13, 2021
Priority date	Sep 11, 2018
Publication date	Jun 20, 2023
Grant date	Jun 20, 2023

Abstract

This disclosure relates to various implementations that dynamically adjust one or more shallow depth of field (SDOF) parameters based on a designated, artificial aperture value. The implementations obtain a designated, artificial aperture value that modifies an initial aperture value for an image frame. The designated, artificial aperture value generates a determined amount of synthetically-produced blur within the image frame. The implementations determine an aperture adjustment factor based on the designated, artificial aperture value in relation to a default so-called “tuning aperture value” (for which the camera's operations may have been optimized). The implementations may then modify, based on the aperture adjustment factor, one or more SDOF parameters for an SDOF operation, which may, e.g., be configured to render a determined amount of synthetic bokeh within the image frame. In response the modified SDOF parameters, the implementations may render an updated image frame that corresponds to the designated, artificial aperture value.

First claim

What is claimed is:

1. A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:
obtain a first image frame comprising a first pixel and second pixel having pixel values;
obtain a blur map that comprises a first blur value for the first pixel and a second blur value for the second pixel;
determine the second pixel is a candidate pixel for the first pixel based on the first blur value;
determine an inclusion weight parameter that is indicative of whether the first pixel is within a bokeh shape of the second pixel; and
determine an output value for the first pixel based on the inclusion weight parameter.

2. The non-transitory program storage device of claim 1 , wherein the bokeh shape of the second pixel is distorted based on a designated, artificial aperture value for an output image frame.

3. The non-transitory program storage device of claim 2 , further comprising instructions stored thereon to cause the one or more processors to:
obtain a tuning aperture value associated with the first image frame, wherein the designated, artificial aperture value causes the bokeh shape of the second pixel to become compressed when the designated, artificial aperture value is larger than the tuning aperture value.

4. The non-transitory program storage device of claim 1 , wherein the bokeh shape is a compressed circle.

5. The non-transitory program storage device of claim 2 , wherein the first image frame includes an image center, and wherein distortion of the bokeh shape of the second pixel is based on distance between the first pixel and the image center and the designated, artificial aperture value.

6. The non-transitory program storage device of claim 5 , wherein an amount of distortion applied to the bokeh shape of the second pixel is based, at least in part, on a location of the first pixel with respect to an edge of the first image frame.

7. The non-transitory program storage device of claim 1 , wherein the instructions that cause the one or more processors to determine the inclusion weight parameter further comprise instructions that cause the one or more processors to:
project the first pixel onto the bokeh shape of the second pixel;
align the bokeh shape of the second pixel along a vector formed from the first pixel to an image center of the first image frame; and
compute a distance value from the first pixel to a center of the bokeh shape of the second pixel.

8. The non-transitory program storage device of claim 7 , wherein the distance value from the first pixel to the center of the bokeh shape of the second pixel indicates whether the first pixel is within a bokeh shape of the second pixel.

9. An imaging system, comprising:
an image capture device;
a programmable control device; and
a memory coupled to the programmable control device, wherein instructions are stored in the memory, and wherein the instructions, when executed, cause the programmable control device to:
obtain a first image frame from the image capture device, the first image frame comprising at least a first pixel and second pixel having pixel values;
obtain a blur map that comprises a first blur value for the first pixel and a second blur value for the second pixel;
determine the second pixel is a candidate pixel for the first pixel based on the first blur value;
determine an inclusion weight parameter that is indicative of whether the first pixel is within a bokeh shape of the second pixel; and
determine an output value for the first pixel based on the inclusion weight parameter.

10. The imaging system of claim 9 , wherein the bokeh shape of the second pixel is distorted based on a designated, artificial aperture value for an output image frame.

11. The imaging system of claim 10 , further comprising instructions stored in the memory to cause the programmable control device to:
obtain a tuning aperture value associated with the first image frame, wherein the designated, artificial aperture value causes the bokeh shape of the second pixel to become compressed when the designated, artificial aperture value is larger than the tuning aperture value.

12. The imaging system of claim 10 , wherein the first image frame includes an image center, and wherein distortion of the bokeh shape of the second pixel is based on distance between the first pixel and the image center and the designated, artificial aperture value.

13. The imaging system of claim 12 , wherein an amount of distortion applied to the bokeh shape of the second pixel is based, at least in part, on a location of the first pixel with respect to an edge of the first image frame.

14. The imaging system of claim 9 , wherein the instructions to determine the inclusion weight parameter further comprise instructions that cause the programmable control device to:
project the first pixel onto the bokeh shape of the second pixel;
align the bokeh shape of the second pixel along a vector formed from the first pixel to an image center of the first image frame; and
compute a distance value from the first pixel to a center of the bokeh shape of the second pixel.

15. An image processing method, comprising:
obtaining a first image frame from the image capture device, the first image frame comprising at least a first pixel and second pixel having pixel values;
obtaining a blur map that comprises a first blur value for the first pixel and a second blur value for the second pixel;
determining the second pixel is a candidate pixel for the first pixel based on the first blur value;
determining an inclusion weight parameter that is indicative of whether the first pixel is within a bokeh shape of the second pixel; and
determining an output value for the first pixel based on the inclusion weight parameter.

16. The method of claim 15 , wherein the bokeh shape of the second pixel is distorted based on a designated, artificial aperture value for an output image frame.

17. The method of claim 16 , further comprising:
obtaining a tuning aperture value associated with the first image frame, wherein the designated, artificial aperture value causes the bokeh shape of the second pixel to become compressed when the designated, artificial aperture value is larger than the tuning aperture value.

18. The method of claim 16 , wherein the first image frame includes an image center, and wherein distortion of the bokeh shape of the second pixel is based on distance between the first pixel and the image center and the designated, artificial aperture value.

19. The method of claim 18 , wherein an amount of distortion applied to the bokeh shape of the second pixel is based, at least in part, on a location of the first pixel with respect to an edge of the first image frame.

20. The method of claim 15 , wherein determining the inclusion weight parameter further comprises:
projecting the first pixel onto the bokeh shape of the second pixel;
aligning the bokeh shape of the second pixel along a vector formed from the first pixel to an image center of the first image frame; and
computing a distance value from the first pixel to a center of the bokeh shape of the second pixel.

Assignees

• Apple Inc

Inventors

• Seely Richard D
• Naaman Alexandre
• Shehane Patrick
• Souza Dos Santos Andre
• Manzari Behkish J

Classifications

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06T5/003 Primary

  Physics · mapped topic

• G06V30/19127

  Extracting features by transforming the feature space, e.g. multidimensional scaling; Mappings, e.g. subspace methods · CPC title

• G06T2200/21

  involving computational photography · CPC title

• G06T7/11

  Region-based segmentation · CPC title