Method and device for balancing foreground-background luminosity

Patent metadata

Field	Value
Publication number	US-11297256-B2
Application number	US-202017061938-A
Country	US
Kind code	B2
Filing date	Oct 2, 2020
Priority date	Jun 2, 2017
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

In one embodiment, a method includes: obtaining a first image of a scene while an illumination component is set to an inactive state; obtaining a second image of the scene while the illumination component is set to a pre-flash state; determining one or more illumination control parameters for the illumination component for a third image of the scene that satisfy a foreground-background balance criterion based on a function of the first and second images in order to discriminate foreground data from background data within the scene; and obtaining the third image of the scene while the illumination component is set to an active state in accordance with the one or more illumination control parameters.

First claim

What is claimed is:

1. A method comprising:
at a device with one or more processors, non-transitory memory, an image sensor, and an illumination component:
obtaining, by the image sensor, a first image of a scene while the illumination component is set to an inactive state;
obtaining, by the image sensor, a second image of the scene while the illumination component is set to a pre-flash state;
determining a macro luminance difference between the first image and the second image and a weighted macro luminance difference between the first image and the second image, wherein, in determining the weighted macro luminance difference, areas with a greater luminance difference between the first image and the second image are weighted greater;
determining, based on the macro luminance difference and the weighted macro luminance difference, one or more illumination control parameters for the illumination component for a third image of the scene that satisfy a foreground luminosity criterion and a foreground-and-background luminosity criterion; and
obtaining, by the image sensor, the third image of the scene while the illumination component is set to an active state in accordance with the one or more illumination control parameters.

2. The method of claim 1 , wherein the foreground luminosity criterion is based on the weighted macro luminance difference and the foreground-and-background luminosity criterion is based on the macro luminance difference.

3. The method of claim 1 , wherein the foreground-and-background luminosity criterion includes a foreground-background balance criterion.

4. The method of claim 1 , wherein the foreground luminosity criterion and the foreground-and-background luminosity criterion are based on predetermined target values.

5. The method of claim 1 , wherein determining the one or more illumination control parameters includes determining a flash period of the illumination component for the third image based on an exposure period of the third image.

6. The method of claim 5 , further comprising determining the exposure period of the third image based on a gain for the third image.

7. The method of claim 6 , further comprising determining the gain for the third image based on a noise reduction heuristic.

8. The method of claim 1 , wherein, in determining the weighted macro luminance difference, areas with a greater luminance difference between the first image and the second image are weighted greater according to a linear model.

9. The method of claim 1 , wherein, in determining the weighted macro luminance difference, areas with a greater luminance difference between the first image and the second image are weighted greater according to a logarithmic model.

10. An electronic device comprising:
an image sensor;
an illumination component;
a non-transitory memory;
one or more processors; and
one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for:
obtaining, by the image sensor, a first image of a scene while the illumination component is set to an inactive state;
obtaining, by the image sensor, a second image of the scene while the illumination component is set to a pre-flash state;
determining a macro luminance difference between the first image and the second image and a weighted macro luminance difference between the first image and the second image, wherein, in determining the weighted macro luminance difference, areas with a greater luminance difference between the first image and the second image are weighted greater;
determining, based on the macro luminance difference and the weighted macro luminance difference, one or more illumination control parameters for the illumination component for a third image of the scene that satisfy a foreground luminosity criterion and a foreground-and-background luminosity criterion; and
obtaining, by the image sensor, the third image of the scene while the illumination component is set to an active state in accordance with the one or more illumination control parameters.

11. The electronic device of claim 10 , wherein the foreground luminosity criterion is based on the weighted macro luminance difference and the foreground-and-background luminosity criterion is based on the macro luminance difference.

12. The electronic device of claim 10 , wherein the foreground-and-background luminosity criterion includes a foreground-background balance criterion.

13. The electronic device of claim 10 , wherein the foreground luminosity criterion and the foreground-and-background luminosity criterion are based on predetermined target values.

14. The electronic device of claim 10 , wherein determining the one or more illumination control parameters includes determining a flash period of the illumination component for the third image based on an exposure period of the third image.

15. The electronic device of claim 14 , further comprising determining the exposure period of the third image based on a gain for the third image.

16. The electronic device of claim 15 , further comprising determining the gain for the third image based on a noise reduction heuristic.

17. The electronic device of claim 10 , wherein, in determining the weighted macro luminance difference, areas with a greater luminance difference between the first image and the second image are weighted greater according to a linear model.

18. The electronic device of claim 10 , wherein, in determining the weighted macro luminance difference, areas with a greater luminance difference between the first image and the second image are weighted greater according to a logarithmic model.

19. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by one or more processors of an electronic device with an image sensor and an illumination component, cause the electronic device to:
obtain, by the image sensor, a first image of a scene while the illumination component is set to an inactive state;
obtain, by the image sensor, a second image of the scene while the illumination component is set to a pre-flash state;
determine a macro luminance difference between the first image and the second image and a weighted macro luminance difference between the first image and the second image, wherein, in determining the weighted macro luminance difference, areas with a greater luminance difference between the first image and the second image are weighted greater;
determine, based on the first image and the second image, one or more illumination control parameters for the illumination component for a third image of the scene that satisfy a foreground luminosity criterion and a foreground-and-background luminosity criterion; and
obtain, by the image sensor, the third image of the scene while the illumination component is set to an active state in accordance with the one or more illumination control parameters.

20. The non-transitory computer-readable medium of claim 19 , wherein the foreground luminosity criterion and the foreground-and-background luminosity criterion are based on predetermined target values.

Assignees

• Apple Inc

Inventors

• Gatt Alexis
• Sachs Todd S
• Johnson Garrett M
• Moelgaard Claus

Classifications

• G06T7/11 Primary

  Region-based segmentation · CPC title

• H04N23/74 Primary

  by influencing the scene brightness using illuminating means · CPC title

• H04N23/73

  by influencing the exposure time · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title