Method for generating depth information and electronic device supporting the same

Patent metadata

Field	Value
Publication number	US-11252306-B2
Application number	US-201916386855-A
Country	US
Kind code	B2
Filing date	Apr 17, 2019
Priority date	Apr 19, 2018
Publication date	Feb 15, 2022
Grant date	Feb 15, 2022

Abstract

An electronic device is provided. The electronic device includes a first camera, a second camera spaced apart from the first camera, and a processor. The processor is configured to obtain a first image of external objects using the first camera, obtain a second image of the external objects using the second camera, identify a specified object in which pieces of depth information are generated from among the external objects included in the first image and the second image based on phase difference comparison between the first image and the second image, then select depth information about the specified object among the pieces of depth information based on a degree of spreading of a point corresponding to the specified object included in at least one of the first image and the second image, and generate depth information about the external objects including the specified object using the selected depth information.

First claim

What is claimed is:

1. An electronic device comprising:
a first camera disposed on one surface of the electronic device;
a second camera disposed on the one surface and spaced apart from the first camera by a specified interval; and
at least one processor, wherein the at least one processor is configured to:
obtain an image using the first camera,
obtain a first image of one or more external objects using the first camera,
obtain a second image of the one or more external objects using the second camera,
obtain depth information on the one or more external objects based on a phase difference comparison between the first image and the second image,
when a specified object among the one or more external objects has pieces of depth information, select depth information from among the pieces of depth information based on a degree of spreading of a point corresponding to the specified object and using a defocus distance, a degree of spreading of a point corresponding to the specified object included in the first image, and a degree of spreading of a point corresponding to the specified object included in the image from among a plurality of degrees of spreading of a point corresponding to the first camera, and

generate a depth map for the one or more external objects using the selected depth information,
wherein the image is obtained before the first image is obtained.

2. The electronic device of claim 1 , wherein the at least one processor is further configured to:
select the depth information of the pieces of depth information by further using at least:
a degree of spreading of a point, which corresponds to the specified object included in the first image, from among a plurality of degrees of spreading of a first point corresponding to the first camera, and

a degree of spreading of a point, which corresponds to the specified object included in the second image, from among a plurality of degrees of spreading of a second point corresponding to the second camera.

3. The electronic device of claim 2 , wherein the at least one processor is further configured to:
identify a first similarity between the plurality of degrees of spreading of the first point and characteristics of the specified object included in the first image and a second similarity between the plurality of degrees of spreading of the second point and characteristics of the specified object included in the second image, and
select the depth information from among the pieces of depth information by further using the first similarity and the second similarity.

4. The electronic device of claim 3 , wherein the at least one processor is further configured to:
select the depth information from among the pieces of depth information by further using the defocus distance corresponding to one of:
a degree of spreading of the first point, in which a sum of the first similarity and the second similarity is smallest, from among the plurality of degrees of spreading of the first point, or
a degree of spreading of the second point, in which a sum of the first similarity and the second similarity is smallest, from among the plurality of degrees of spreading of the second point.

5. The electronic device of claim 3 , wherein the at least one processor is further configured to:
in the identifying of the first similarity and the second similarity, use a degree of spreading of the first point and a degree of spreading of the second point, which have a same defocus distance, from among the plurality of degrees of spreading of the first point and the plurality of degrees of spreading of the second point.

6. The electronic device of claim 2 ,
wherein the plurality of degrees of spreading of the first point or the plurality of degrees of spreading of the second point comprise depth relationship information about change in a degree of spreading of a point according to change of at least part of a distance between the electronic device and the one or more external objects, and
wherein the at least one processor is further configured to select the depth information by further using the depth relationship information.

7. The electronic device of claim 1 , wherein the at least one processor is further configured to:
identify the degree of spreading of the point corresponding to the specified object included in the first image and the degree of spreading of the point corresponding to the specified object included in the third image using degrees of spreading of the point, which are spaced apart from each other by a defocus distance between the first image and the third image, from among the plurality degrees of spreading of the point, and
select the depth information from among the pieces of depth information by further using a defocus distance of the degree of spreading of the point corresponding to the specified object included in the first image.

8. The electronic device of claim 1 ,
wherein the first image comprises a plurality of external objects,
wherein the second image comprises a plurality of external objects, and
wherein the at least one processor is further configured to:
extract a first region including a specified object among a whole region of the first image and identify a first characteristic of the extracted first region, wherein the first characteristic comprises data obtained by converting the image of the first region into a first frequency domain, and
extract a second region including a specified object among a whole region of the second image and identify a second characteristic of the extracted second region, wherein the second characteristic comprises data obtained by converting the image of the second region into a second frequency domain.

9. The electronic device of claim 8 , wherein the at least one processor is further configured to:
identify the specified object from the specified object of the first image and the specified object of the second image using information of the first frequency domain and information of the second frequency domain.

10. An electronic device comprising:
a camera;
a memory; and
at least one processor,
wherein the camera comprises a first group of pixels and a second group of pixels per pixel,
wherein the at least one processor is configured to:
obtain an image using the camera,
obtain a first image of one or more external objects using the first group of pixels of the camera,
obtain a second image of the one or more external objects using the second group of pixels of the camera,
obtain depth information on the one or more external objects based on a phase difference comparison between the first image and the second image,
when a specified object among the one or more external objects has pieces of depth information, select depth information from among the pieces of depth information based on a degree of spreading of a point of the specified object and using a defocus distance, a degree of spreading of a point, which corresponds to the specified object included in the first image, from among a plurality of degrees of spreading of a point corresponding to the camera, and a degree of spreading of a point which corresponds to the specified object included in the image, and

generate a depth map for the one or more external objects using the selected depth information, and
wherein the image is obtained before the first image is obtained.

11. The electronic device of claim 10 , wherein the at least one processor is further configured to:
select the depth information of the pieces of depth information by further using at least:
a degree of spreading of a point, which corresponds to the specified object included in the first image, from among a plurality of d

Assignees

• Samsung Electronics Co Ltd

Inventors

• YANG Kyungdong
• Park Jaehyoung
• Yu Yungmok
• Choi Dongbum
• Won Jonghun

Classifications

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• H04N5/2226 Primary

  Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10) · CPC title

• G06T7/593

  from stereo images · CPC title

• G06T7/571

  from focus · CPC title

• G06T2207/10012

  Stereo images · CPC title