Variable aperture (VA) camera control for controlling a depth of focus

Patent metadata

Field	Value
Publication number	US-12008763-B2
Application number	US-202217817584-A
Country	US
Kind code	B2
Filing date	Aug 4, 2022
Priority date	Aug 4, 2022
Publication date	Jun 11, 2024
Grant date	Jun 11, 2024

Abstract

This disclosure provides systems, methods, and devices for image capture and image processing that support capture using a variable aperture (VA) camera. In a first aspect, a method of image processing includes determining a plurality of depths for a scene, the plurality of depths corresponding to a plurality of regions of interest; determining an aperture size for a camera based on the plurality of depths; determining a lens position for the camera based on the aperture size and the plurality of depths; and controlling the camera to adjust a variable aperture (VA) of the camera based on the aperture size and to adjust a lens associated with the variable aperture (VA) based on the lens position. Other aspects and features are also claimed and described.

First claim

What is claimed is:

1. A method, comprising:
determining a plurality of depths for a scene and a plurality of regions of interest, wherein a first one of the plurality of regions of interest corresponds to a first one of the plurality of depths, which is at a first distance from a reference point, and a second one of the plurality of regions of interest corresponds to a second one of the plurality of depths, which is at a second distance from the reference point;
determining an aperture size for a camera based on a difference between the first distance and the second distance;
determining a lens position for the camera that provides a focal plane that is between the first distance and the second distance based on the aperture size and the first one of the plurality of depths and the second one of the plurality of depths; and
controlling the camera to adjust a variable aperture (VA) of the camera based on the determined aperture size and to adjust a lens associated with the variable aperture (VA) based on the determined lens position.

2. The method of claim 1 , wherein determining the aperture size comprises determining an aperture size having a depth of focus at least as large as the difference between the first distance and the second distance.

3. The method of claim 2 , wherein determining the lens position comprises determining a lens position such that the first one of the plurality of depths and the second one of the plurality of depths are both within the depth of focus extending around a focal plane of the lens position.

4. The method of claim 1 , wherein determining the plurality of regions of interest comprises determining locations of at least two faces in the scene, wherein the first of the plurality of depths and the second of the plurality of depths correspond to respective depths at the locations of the at least two faces.

5. The method of claim 1 , wherein determining the aperture size for the camera comprises:
determining an initial aperture size based on light conditions in the scene; and
adjusting the initial aperture size to determine the aperture size based on the difference between the first distance and the second distance.

6. The method of claim 1 , wherein determining the plurality of depths for a scene comprises receiving depth data for the scene from a depth sensor.

7. The method of claim 1 , further comprising:
receiving image data from the camera, the image data captured at the variable aperture and the lens position; and
determining at least one image frame based on the image data.

8. The method of claim 7 , further comprising:
determining a camera depth of focus for the camera at the lens position and at the aperture size is less than a distance between a first depth of the plurality of depths and a second depth of the plurality of depths, wherein the determining the at least one image frame comprises deblurring at least a portion of one region of interest of the plurality of regions of interest that is beyond the camera depth of focus.

9. The method of claim 1 , further comprising:
receiving first image data representing the scene from the camera, wherein the determining of the plurality of regions of interest comprises determining a plurality of locations of a plurality of objects of the scene by performing object detection on the first image data.

10. The method of claim 9 , further comprising:
receiving user input to specify the plurality of regions of interest from the plurality of locations, wherein the first one of the plurality of depths and the second one of the plurality of depths correspond to objects in the scene selected by a user.

11. The method of claim 10 , wherein determining the plurality of locations of a plurality of objects of the scene comprises determining a plurality of locations of a plurality of faces of the scene by performing facial recognition on the first image data.

12. The method of claim 10 , wherein determining the plurality of locations of a plurality of objects of the scene comprises determining a plurality of locations by applying a saliency model to the first image data to determine locations having a salience score above a threshold value.

13. The method of claim 10 , further comprising:
receiving second image data from the camera after controlling the camera to adjust a variable aperture (VA) of the camera based on the aperture size and to adjust a lens associated with the variable aperture (VA) based on the lens position, the second image data captured based on the aperture size and the lens position; and
determining at least one image frame based on the second image data.

14. An apparatus, comprising:
a memory storing processor-readable code; and
at least one processor coupled to the memory, the at least one processor configured to execute the processor-readable code to cause the at least one processor to perform operations including:
determining a plurality of depths for a scene and a plurality of regions of interest, wherein a first one of the plurality of regions of interest corresponds to a first one of the plurality of depths, which is at a first distance from a reference point, and a second one of the plurality of regions of interest corresponds to a second one of the plurality of depths, which is at a second distance from the reference point;
determining an aperture size for a camera based on a difference between the first distance and the second distance;
determining a lens position for the camera that provides a focal plane that is between the first distance and the second distance based on the aperture size and the first one of the plurality of depths and the second one of the plurality of depths; and
controlling the camera to adjust a variable aperture (VA) of the camera based on the determined aperture size and to adjust a lens associated with the variable aperture (VA) based on the determined lens position.

15. The apparatus of claim 14 , wherein determining the lens position comprises determining a lens position such that the first one of the plurality of depths and the second one of the plurality of depths are both within the depth of focus extending around a focal plane of the lens position.

16. The apparatus of claim 14 , wherein determining the aperture size comprises determining an aperture size having a depth of focus at least as large as the different between the first distance and the second distance.

17. The apparatus of claim 14 , wherein determining the plurality of regions of interest comprises determining locations of at least two faces in the scene, wherein the first of the plurality of depths and the second of the plurality of depths correspond to respective depths at the locations of the at least two faces.

18. The apparatus of claim 14 , wherein determining the aperture size for the camera comprises:
determining an initial aperture size based on light conditions in the scene; and
adjusting the initial aperture size to determine the aperture size based on the difference between the first distance and the second distance.

19. The apparatus of claim 14 , wherein determining the plurality of depths for a scene comprises receiving a depth map for the scene from a depth sensor.

20. The apparatus of claim 14 , wherein the processor is further configured to perform operations comprising:
receiving image data from the camera, the image data captured at the variable aperture and the lens position; and
determining at least one image frame based on the image data.

21. The apparat

Assignees

• Qualcomm Inc

Inventors

• Feng Wen-Chun
• Galor Gluskin Micha
• Kao Hui Shan
• Liaw Hang-Wei

Classifications

• H04N23/611 Primary

  where the recognised objects include parts of the human body · CPC title

• H04N23/959

  by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G03B9/00

  Exposure-making shutters; Diaphragms · CPC title

• H04N5/2226

  Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10) · CPC title