Electronic device, method, and computer-readable medium for providing bokeh effect in video

Patent metadata

Field	Value
Publication number	US-11258962-B2
Application number	US-202016840135-A
Country	US
Kind code	B2
Filing date	Apr 3, 2020
Priority date	Apr 4, 2019
Publication date	Feb 22, 2022
Grant date	Feb 22, 2022

Abstract

An electronic device includes a plurality of cameras, a memory, and at least one processor. The at least one processor is configured to identify a change in a distance between a focused subject and the electronic device within a first frame through at least some of the plurality of cameras while a video with a bokeh effect is acquired. The at least one processor is also configured to provide the bokeh effect of the video within a first background area in a second frame in response to identifying that a difference between a first distance and a second distance is within a criterion range. The at least one processor is further configured to provide the bokeh effect of the video within a second background area identified based on a third distance in the second frame in response to identifying that the difference is out of the criterion range.

First claim

What is claimed is:

1. An electronic device comprising:
a plurality of cameras;
a memory configured to store instructions; and
at least one processor,
wherein the at least one processor is configured to:
identify a change in a distance between a focused subject and the electronic device from a first distance to a second distance within a first frame through at least some of the plurality of cameras while a video with a bokeh effect is acquired using the plurality of cameras,
provide the bokeh effect of the video within a first background area identified based on the second distance in a second frame subsequence to the first frame in response to identifying that a difference between the first distance and the second distance is within a criterion range, and
provide the bokeh effect of the video within a second background area identified based on a third distance between the first distance and the second distance in the second frame in response to identifying that the difference is out of the criterion range.

2. The electronic device of claim 1 , wherein the at least one processor is further configured to: provide the bokeh effect of the video within a third background identified based on a fourth distance between the second distance and the third distance in a third frame subsequent to the second frame.

3. The electronic device of claim 1 , wherein the at least one processor is further configured to: provide the bokeh effect of the video in a third background area identified based on the first distance in the second frame in response to identifying that the difference is out of the criterion range and another criterion range.

4. The electronic device of claim 1 , wherein the plurality of cameras comprise a Red-Green-Blue (RGB) camera and a depth camera.

5. The electronic device of claim 4 , wherein the depth camera comprises:
an emitter configured to emit an infrared light, and
a receiver configured to receive a reflected light of the infrared light.

6. The electronic device of claim 5 , wherein the at least one processor is configured to:
acquire first data on a distance between each of a plurality of subjects including the focused subject in a Field of View (FOV) of the depth camera and the electronic device, based on the reflected light,
acquire third data refined from the first data using second data acquired through the RGB camera in response to identifying that some of the plurality of subjects are out of a maximum distance from the electronic device that can be measured using the depth camera, based on some of the first data, and
identify that the distance between the focused subject and the electronic device is changed from the first distance to the second distance within the first frame, based on the third data.

7. The electronic device of claim 6 , wherein the at least one processor is configured to:
acquire the third data refined from the first data by converting some of the first data to data corresponding to a distance between remaining parts of the focused subject and the electronic device, using the second data.

8. The electronic device of claim 4 , further comprising a display,
wherein the at least one processor is configured to identify the focused subject among a plurality of subjects in a Field of View (FOV) of the depth camera, based on a criterion having a highest priority among a plurality of criteria, wherein the plurality of criteria include:
a first criterion to identify a subject designated by a touch input received through the display among the plurality of subjects as the focused subject,
a second criterion to identify a subject corresponding to a face among the plurality of subjects as the focused subject, and
a third criterion to identify at least one subject included in an Auto Focus (AF) area among the plurality of subjects and identify a subject closest to the electronic device among the at least one identified subject as the focused subject,

wherein a priority of the first criterion is higher than a priority of the second criterion and a priority of the third criterion, and
wherein the priority of the second criterion is higher than the priority of the third criterion.

9. The electronic device of claim 4 , wherein the at least one processor is further configured to:
determine that the focused subject moves out of a maximum distance from the electronic device that is measurable using the depth camera through at least some of the plurality of cameras while the video with the bokeh effect is acquired using the plurality of cameras, and
gradually remove the bokeh effect from the video, based on the determination.

10. The electronic device of claim 1 , wherein the at least one processor is configured to:
identify the third distance by adding a criterion distance to the first distance in response to identifying that the difference is out of the criterion range,
provide the bokeh effect of the video within the second background area identified based on the third distance in the second frame,
identify a fourth distance by adding the criterion distance to the third distance, and
provide the bokeh effect of the video within a third background area identified based on the fourth distance in a third frame subsequent to the second frame.

11. A method performed by an electronic device with a plurality of cameras, the method comprising:
identifying a change in a distance between a focused subject and the electronic device from a first distance to a second distance within a first frame through at least some of the plurality of cameras while a video with a bokeh effect is acquired using the plurality of cameras;
providing the bokeh effect of the video within a first background area identified based on the second distance in a second frame subsequence to the first frame in response to identifying that a difference between the first distance and the second distance is within a criterion range; and
providing the bokeh effect of the video within a second background area identified based on a third distance between the first distance and the second distance in the second frame in response to identifying that the difference is out of the criterion range.

12. The method of claim 11 , further comprising providing the bokeh effect of the video within a third background identified based on a fourth distance between the second distance and the third distance in a third frame subsequent to the second frame.

13. The method of claim 11 , further comprising providing the bokeh effect of the video in a third background area identified based on the first distance in the second frame in response to identifying that the difference is out of the criterion range and another criterion range.

14. The method of claim 11 , wherein:
the plurality of cameras comprises a Red-Green-Blue (RGB) camera and a depth camera, the depth camera includes an emitter for emitting an infrared light, and a receiver for receiving a reflected light of the infrared light, and
identifying the change in the distance from the first distance to the second distance within the first frame comprises:
acquiring first data on a distance between each of a plurality of subjects including the focused subject in a Field of View (FOV) of the depth camera and the electronic device, based on the reflected light,
acquiring third data refined from the first data using second data acquired through the RGB camera in response to identifying that some of the plurality of subjects are out of a maximum distance from the electronic device that can be measured using the depth camera, based on some of the first data, and
identifying that the distance

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Yumin
• Kim Junghoon
• Lee Sungho
• Hong Taehwa
• Park Jiyoon

Classifications

• H04N23/959 Primary

  by adjusting depth of field during image capture, e.g. maximising or setting range based on scene characteristics · CPC title

• H04N23/67

  Focus control based on electronic image sensor signals · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/45

  for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images · CPC title

• G03B13/20

  Rangefinders coupled with focusing arrangements, e.g. adjustment of rangefinder automatically focusing camera · CPC title