Electronic devices and corresponding methods for creating and presenting altered image sensor data simulating a bokeh effect

Patent metadata

Field	Value
Publication number	US-12450701-B2
Application number	US-202318215646-A
Country	US
Kind code	B2
Filing date	Jun 28, 2023
Priority date	Jun 28, 2023
Publication date	Oct 21, 2025
Grant date	Oct 21, 2025

Abstract

An electronic device includes an image capture device configured to capture image sensor data in response to initiation of an image capture operation. The electronic device includes one or more processors and a display operable with the one or more processors. The one or more processors select saturated pixels of the image sensor data and determine a gain factor as a function of brightness values of pixels in the image sensor data. The one or more processors increase an intensity value of the saturated pixels by the gain factor to create scaled image sensor data, blur at least some of the scaled image sensor data by convolving a blur kernel with the scaled image sensor data to create altered image sensor data exhibiting a simulated bokeh effect and cause the display to present the altered image sensor data exhibiting the simulated bokeh effect.

First claim

What is claimed is:

1. A method for an electronic device, the method comprising:
receiving, by one or more processors from an image sensor of an image capture device, image sensor data;
increasing, by the one or more processors, an intensity value of saturated pixels by a gain factor determined as a function of brightness values of pixels of the image sensor data;
blurring, by the one or more processors, at least some of the image sensor data by convolving a blur kernel having a predefined shape with the image sensor data to create altered image sensor data exhibiting a simulated bokeh effect; and
presenting, by the one or more processors, the altered image sensor data on a user interface of the electronic device.

2. The method of claim 1 , wherein the predefined shape is a projection of a lens aperture of the image capture device.

3. The method of claim 2 , wherein the predefined shape comprises a pixelated circle.

4. The method of claim 1 , wherein the predefined shape comprises one of a pixelated heart, a pixelated star, or a pixelated snowflake.

5. The method of claim 1 , wherein:
the blur kernel has a dimension of N pixels by N pixels; and
the blurring comprises performing a quick convolution consisting of a number of product operations proportional to 2*N and less than N*N.

6. The method of claim 1 , wherein:
the predefined shape spans a plurality of pixels of the image sensor data; and
the convolving the blur kernel comprises translating the predefined shape across the image sensor data while:
adding convolution product values for a first set of pixels entering the predefined shape when the blur kernel translates;
subtracting convolution product values for a second set of pixels exiting the predefined shape when the blur kernel translates; and
without recalculating product values of a third set of pixels remaining within the predefined shape when the blur kernel translates.

7. The method of claim 6 , wherein the gain factor is greater than or equal to one.

8. The method of claim 7 , wherein the gain factor has a formula of:
G=K1*Bv+K3, where:
G represents the gain factor;
Bv represents a brightness value associated with the saturated pixels:
K1 and K3 are tunable constants; and
when G is less than one, G is defined to be one.

9. The method of claim 8 , where Bv represents an average brightness value for a predefined number of pixels surrounding, and including, each saturated pixel.

10. The method of claim 9 , further comprising determining, by the one or more processors, the average brightness value by creating a summed area table for the predefined number of pixels surrounding, and including, the each saturated pixel.

11. The method of claim 9 , wherein:
the saturated pixels each have a luma value greater than a predefined minimum luma value threshold; and
the predefined number of pixels surrounding, and including, the each saturated pixel is less than a predefined maximum saturated pixel count threshold.

12. The method of claim 9 , further comprising, by the one or more processors before the increasing the intensity value of the saturated pixels by the gain factor:
downscaling a luma (Y) channel of the image sensor data representing perceived intensity of pixels of the image sensor data;
selecting the saturated pixels by determining which pixels of the image sensor data have a luma value greater than a predefined minimum luma value threshold; and
determining the average brightness value by creating a summed area table for the predefined number of pixels surrounding, and including, the each saturated pixel.

13. The method of claim 12 , further comprising, by the one or more processors after the blurring the at least some of the image sensor data:
upscaling the luma (Y) channel of the altered image sensor data; and
blurring at least some other of the image sensor data by:
creating other summed area tables for a blue projection (Cb) channel of the altered image sensor data and a red projection (Cr) channel of the altered image sensor data; and
applying a box filter to the other summed area tables.

14. The method of claim 1 , wherein the presenting the altered image sensor data occurs while the image capture device is operating in a preview mode of operation.

15. An electronic device, comprising:
an image capture device configured to capture image sensor data in response to initiation of an image capture operation;
one or more processors; and
a display operable with the one or more processors;
wherein the one or more processors:
select saturated pixels of the image sensor data;
determine a gain factor as a function of brightness values of pixels in the image sensor data;
increase an intensity value of the saturated pixels by the gain factor to create scaled image sensor data;
blur at least some of the scaled image sensor data by convolving a blur kernel with the scaled image sensor data to create altered image sensor data exhibiting a simulated bokeh effect; and
cause the display to present the altered image sensor data exhibiting the simulated bokeh effect.

16. The electronic device of claim 15 , wherein the image sensor data depicts a low-light image, and the saturated pixels depict lights within a field of view of the image capture device.

17. The electronic device of claim 16 , wherein the one or more processors blur the at least some of the scaled image sensor data by convolving a CIRC function with the scaled image sensor data.

18. A method for an electronic device, the method comprising:
receiving, by one or more processors from an image sensor of an image capture device, image sensor data;
selecting, by the one or more processors, saturated pixels of the image sensor data by determining which pixels of the image sensor data have a luma value greater than a predefined minimum luma value threshold;
determining, by the one or more processors, an average brightness value for a predefined number of pixels surrounding, and including, each saturated pixel;
determining, by the one or more processors, a gain factor from the average brightness value;
increasing, by the one or more processors, an intensity value of the saturated pixels by the gain factor to create scaled image sensor data;
blurring, by the one or more processors, at least some of the scaled image sensor data by convolving a blur kernel with the scaled image sensor data to create altered image sensor data exhibiting a simulated bokeh effect; and
presenting, by the one or more processors, the altered image sensor data on a user interface of the electronic device.

19. The method of claim 18 , further comprising:
by the one or more processors before the selecting the saturated pixels of the image sensor data, downscaling a luma (Y) channel of the image sensor data representing perceived intensity of pixels of the image sensor data; and
by the one or more processors after the blurring the at least some of the scaled image sensor data, upscaling the luma (Y) channel of the altered image sensor data.

20. The method of claim 19 , further comprising:
blurring at least some other of the image sensor data by:
creating other summed area tables for a blue projection (Cb) channel of the altered image sensor data and a red projection (Cr) channel of the altered image sensor data; and
applying a box filter to the other summed area tables.

Assignees

• Motorola Mobility Llc

Inventors

• Yang Emily
• Pincenti John

Classifications

• H04N23/71 Primary

  Circuitry for evaluating the brightness variation · CPC title

• G06T5/20

  using local operators · CPC title

• H04N23/85

  for matrixing · CPC title

• G06T2207/10024

  Color image · CPC title

• H04N23/86

  for controlling the colour saturation of colour signals, e.g. automatic chroma control circuits · CPC title