Electronic device and method for controlling the electronic device

Patent metadata

Field	Value
Publication number	US-12413842-B2
Application number	US-202318136148-A
Country	US
Kind code	B2
Filing date	Apr 18, 2023
Priority date	Oct 21, 2020
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

A method performed by an electronic device, includes: A method for performed by an electronic device, includes: acquiring an image by using a first lens; inputting information about a plurality of candidate regions of the image into a neural network model that is trained to acquire a score indicating whether a face of a person is included, and acquiring scores for the plurality of candidate regions; identifying a candidate region including at least part of a person's face among the plurality of candidate regions based on the acquired scores; and providing guide information for changing a capturing composition based on information about the location of the identified person region and a score of the identified person region.

First claim

What is claimed is:

1. A method for performed by an electronic device, the method comprising:
acquiring an image by using a first lens;
inputting information about a plurality of candidate regions of the image into a neural network model that is trained to acquire a score indicating whether an entire face of a person is captured in a candidate region, and acquiring scores for the plurality of candidate regions;
identifying a person region including at least part of a person's face among the plurality of candidate regions based on the acquired scores; and
providing guide information for changing a capturing composition based on information about a location of the identified person region and a score of the identified person region.

2. The method of claim 1 , further comprising dividing an edge portion of the image into the plurality of candidate regions, and acquiring location information of the plurality of candidate regions,
wherein the acquiring the scores comprises:
inputting information about the plurality of candidate regions of the image into the neural network model, and
acquiring information on key points in the plurality of candidate regions together with the scores for the plurality of candidate regions.

3. The method of claim 2 , further comprising acquiring information about a changing direction of the capturing composition based on information about the location of the identified person region, information on key points in the identified person region, and the score of the identified person region.

4. The method of claim 3 , wherein the providing the guide information comprises providing the guide information comprising a message guiding a moving direction of the person in the person region for changing the capturing composition based on the acquired information about the changing direction of the capturing composition.

5. The method of claim 4 , wherein the providing the guide information further comprises providing the guide information in a visual form through a display or providing in an auditory form through a speaker.

6. The method of claim 3 , further comprising providing a first user interface (UI) element indicating a center of the image while acquiring the image,
wherein the providing the guide information comprises providing the guide information comprising a second UI element for guiding movement of the first UI element to a location identified based on the acquired information about the changing direction of the capturing composition.

7. The method of claim 1 , further comprising displaying a user interface (UI) for setting a change of the first lens for changing a capturing composition.

8. The method of claim 7 , further comprising, based on determining that the acquired score is within a threshold range while the setting of the change of the first lens is turned on for changing the capturing composition through the UI, capturing an image by using a second lens different from the first lens based on the information about the location of the identified person region and the score of the identified person region.

9. The method of claim 8 , wherein the second lens is configured to acquire the image of a wider field of view than the first lens.

10. The method of claim 1 , further comprising:
acquiring a reference score acquired by multiplying a number of pixels in face parts of a training image comprising the entire face by weights according to an importance of the face parts;
acquiring a crop score acquired by multiplying a number of pixels in face parts in a crop image which randomly cropped the training image by weights according to the importance of the face parts;
identifying a ratio of the reference score and the crop score as a face score corresponding to the crop image; and
training the neural network model based on the crop image and the face score corresponding to the crop image.

11. An electronic device comprising:
a camera including a first lens;
a display;
a memory; and
a processor configured to:
acquire an image by using the first lens,
input information about a plurality of candidate regions of the image into a neural network model that is trained to acquire a score indicating whether an entire face of a person is captured in a candidate region, and acquire scores for the plurality of candidate regions,
identify a person region including at least part of a person's face among the plurality of candidate regions based on the acquired scores, and
control the display to provide guide information for changing a capturing composition based on information about a location of the identified person region and a score of the identified person region.

12. The electronic device of claim 11 , wherein the processor is further configured to:
divide an edge portion of the image into the plurality of candidate regions,
acquire location information of the plurality of candidate regions,
input information about the plurality of candidate regions of the image into the neural network model, and
acquire information on key points in the plurality of candidate regions with scores for the plurality of candidate regions.

13. The electronic device of claim 12 , wherein the processor is further configured to acquire information about a changing direction of the capturing composition based on information about the location of the identified person region, information on key points in the identified person region, and the score of the identified person region.

14. The electronic device of claim 13 , wherein the processor is further configured to control the display to provide the guide information comprising a message guiding a moving direction of the person in the person region for changing the capturing composition, based on the acquired information about the changing direction of the capturing composition.

15. The electronic device of claim 14 , further comprising a speaker,
wherein the processor is further configured to provide the guide information in an auditory form through the speaker.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Dongchan
• Hwang Jinyoung

Classifications

• H04N23/632

  for displaying or modifying preview images prior to image capturing, e.g. variety of image resolutions or capturing parameters · CPC title

• H04N23/815

  for controlling the resolution by using a single image · CPC title

• H04N23/64 Primary

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/633

  for displaying additional information relating to control or operation of the camera · CPC title

• H04N23/62

  Control of parameters via user interfaces · CPC title