Electronic device including folded camera module for flare compensation and control method thereof

Patent metadata

Field	Value
Publication number	US-12401907-B2
Application number	US-202318324325-A
Country	US
Kind code	B2
Filing date	May 26, 2023
Priority date	May 27, 2022
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

An aspect of the disclosure provides an electronic device. The electronic device may include a lens assembly, an image sensor including multiple unit pixels configured to convert the light passed through the lens assembly into an electrical signal, and a processor electrically connected to the image sensor to output an image based on the electrical signal. At least one unit pixel included in the multiple unit pixels includes a micro-lens and at least two subpixels formed to correspond to the micro-lens, and the processor is configured to acquire multiple electrical signals from the at least two subpixels, identify a ratio of the multiple electrical signals, and determine, based on the identified ratio, whether flare is generated due to refraction or reflection of the light.

First claim

What is claimed is:

1. An electronic device comprising:
a lens assembly, including at least one lens, and at least one reflector or refractor, which has a trapezoid shape, configured to refract or reflect light;
an image sensor including multiple unit pixels configured to convert the light passed through the lens assembly into an electrical signal; and
at least one processor, comprising processing circuitry, electrically connected to the image sensor to output an image based on the electrical signal,
wherein at least a part of the light refracted or reflected by the reflector or refractor is incident on the image sensor along a second optical path different from a first optical path which is incident on the image sensor,
wherein at least one unit pixel included in the multiple unit pixels comprises:
a micro-lens; and
at least two subpixels formed to correspond to the micro-lens and arranged in a direction corresponding to the second optical path, and

wherein the at least one processor is configured to:
acquire multiple electrical signals from the at least two subpixels;
identify a ratio of the multiple electrical signals; and
determine, based on the identified ratio, whether flare is generated due to refraction or reflection of the light.

2. The electronic device of claim 1 , wherein the at least one processor is configured to:
compensate the flare, based on the identified ratio.

3. The electronic device of claim 2 , wherein the at least two subpixels are arranged in a direction corresponding to an incident angle of light incident through the micro-lens.

4. The electronic device of claim 1 , wherein the at least one processor is configured to determine whether the flare is generated due to the refraction or reflection of the light only for a specific area of the image sensor in which the light incident along the second optical path is received.

5. The electronic device of claim 4 , wherein, as the light incident along the second optical path is received, at least one of the at least two subpixels is configured to output a higher gain than other subpixels, and the at least one processor is configured to identify a ratio of multiple electrical signals with reference to the subpixel outputting the higher gain.

6. The electronic device of claim 4 , further comprising a memory configured to pre-store a lookup table (LUT) related to a ratio according to a direction predesignated based on the second optical path.

7. The electronic device of claim 6 , wherein the at least one processor is configured to, when executed, determine whether the flare is generated and compensate the flare using the LUT stored in the memory and the identified ratio.

8. The electronic device of claim 1 , wherein the at least one processor is configured to:
distinguish saturated subpixels based on the multiple electrical signals acquired from the at least two subpixels; and
compensate the flare depending on a number of saturated subpixels.

9. The electronic device of claim 8 , wherein the at least one processor is configured to compensate, in a case that each of the at least two subpixels in one unit pixel are saturated, the flare using a non-saturated unit pixel around the saturated unit pixel.

10. The electronic device of claim 8 , wherein the at least one processor is configured to compensate, in a case that one or more of the at least two subpixels in one unit pixel are saturated, the flare using a non-saturated subpixel, AF (autofocusing) information, or a non-saturated unit pixel around the saturated subpixel.

11. The electronic device of claim 10 , wherein the at least one processor is configured to remove an artifact of the saturated subpixel or an artifact therearound through gamma or clipping calculation after the flare is compensated.

12. The electronic device of claim 1 , wherein the image sensor is configured to have a micro-lens configuration comprising 2PD, 4PD, 9PD, NPD, metal shield, slanted 2PD, or 2×1 OCL PD.

13. A method for controlling an electronic device comprising a camera comprising a lens assembly, at least one reflector or refractor, which has a trapezoid shape, configured to refract or reflect light, and an image sensor comprising multiple unit pixels; and at least one processor comprising processing circuitry, the method comprising:
acquiring multiple electrical signals from at least two subpixels included in each of the multiple unit pixels and arranged in a direction corresponding to a second optical path, based on a ratio of light incident along a first optical path in which the light is refracted or reflected at least twice in the lens assembly and the second optical path different from the first optical path;
identifying a ratio of the multiple electrical signals;
determining, based on the identified ratio, whether a flare is generated by refraction or reflection of the light; and
compensate the flare.

14. The method of claim 13 , further comprising:
determining whether the flare is generated or compensate the flare using a LUT stored in a memory and the identified ratio.

15. The method of claim 13 , further comprising:
distinguishing saturated subpixels based on the multiple electrical signals acquired from the at least two subpixels; and
compensating the flare depending on a number of saturated subpixels.

16. The method of claim 13 , further comprising:
compensating, in a case that all of at least two subpixels in one unit pixel are saturated, the flare using a non-saturated unit pixel around the saturated unit pixel.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Park Jaehyoung
• Won Jonghun

Classifications

• G06T5/80

  Geometric correction · CPC title

• H04N23/55 Primary

  Optical parts specially adapted for electronic image sensors; Mounting thereof · CPC title

• H04N23/71 Primary

  Circuitry for evaluating the brightness variation · CPC title

• H10F39/12

  Image sensors · CPC title

• H04N25/704

  Pixels specially adapted for focusing, e.g. phase difference pixel sets · CPC title