Electronic device for providing shooting mode based on virtual character and operation method thereof

Patent metadata

Field	Value
Publication number	US-12190576-B2
Application number	US-202117411393-A
Country	US
Kind code	B2
Filing date	Aug 25, 2021
Priority date	Feb 19, 2019
Publication date	Jan 7, 2025
Grant date	Jan 7, 2025

Abstract

An electronic device according to an embodiment may include: a camera module; a display; and a processor, wherein the processor may be configured to: obtain a preview image corresponding to an external object using the camera module; determine attributes of the external object, based on the obtained preview image; synthesize the preview image with a virtual character image, based on the attributes of the external object; and output the synthesized preview image through the display. Other embodiments may be provided.

First claim

What is claimed is:

1. An electronic device comprising:
a camera;
a display;
a processor; and
memory storing instructions that, when executed by the processor, cause the electronic device to:
obtain, using the camera, a preview image including an object;
track the object in the obtained preview image;
determine whether or not the object includes a face of a user;
produce an avatar image including an avatar which has features corresponding to the user's face and/or which simulates a motion of the object based on the tracking of the object;
synthesize the avatar image with at least a portion of the object in the preview image;
output, through the display, the preview image such that the avatar image replaces the user's face in the preview image; and
select between providing a facial avatar and providing a body avatar according to whether a first portion or a second portion of a body of the user is a predetermined size relative to the object in the preview image.

2. The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to output a user interface for changing at least one aspect of the avatar image, and
wherein producing the avatar image includes selecting avatar features reflecting the features of the user face for inclusion in the avatar image, such that the produced avatar image resembles the user's face.

3. The electronic device of claim 2 , wherein the instructions, when executed by the processor, cause the electronic device to:
output a plurality of selection items for changing various attribute information of the avatar; and
in response to detecting a selection of one of the selection items from among the plurality of selection items, modify a visual appearance of the avatar by incorporating a selected characteristic into the avatar.

4. The electronic device of claim 3 , wherein the plurality of selection items includes sequential display of a first menu, a second menu and a third menu,
wherein the first menu including selectable items to indicate an applicable sex of the synthesized object, the second menu including selectable items representing applicable facial features of the synthesized object, and the third menu including selectable items to indicate applicable costume of the synthesized object.

5. The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
synthesize at least a portion of the object in the preview image with the avatar image such that the avatar image is displayed together with an image corresponding to the object of the preview image.

6. The electronic device of claim 5 , wherein the instructions when executed by the processor, cause the electronic device to:
identify a face area of the object in the preview image; and
synthesize the object with the avatar image such that at least a face portion of the avatar image replaces the face area of the object.

7. The electronic device of claim 5 , wherein the instructions, when executed by the processor, cause the electronic device to:
identify a whole area of the object in the preview image; and
synthesize the object with the avatar image such that a whole portion of the avatar image replaces the whole area of the object.

8. The electronic device of claim 1 ,
wherein providing the facial avatar is selected when the face is greater than the predetermined size and providing the facial avatar includes replacing the face within the object with a generated face of the avatar image, and
wherein providing the body avatar is selected when the upper body is greater than the predetermined size, and wherein providing the body avatar includes displaying a full body display of the avatar image with the upper body of the user within the object.

9. The electronic device of claim 1 , wherein the instructions, when executed by the processor, cause the electronic device to:
track distinguished features of the object in the obtained preview image; and
produce the avatar image based on the tracked distinguished features.

10. The electronic device of claim 9 , wherein the tracked distinguished features correspond to a face contour, a pupil, a mouth, a hand, a finger, and/or a body included in the object.

11. The electronic device of claim 10 , wherein the instructions, when executed by the processor, cause the electronic device to:
determine a tracking level with respect to the object; and
track the distinguished features of the object based on the determined tracking level.

12. The electronic device of claim 11 , wherein the tracking level is determined based on a distance between the electronic device and the object.

13. The electronic device of claim 11 , wherein the tracking level is determined based on a size of an imaging area corresponding to the object in the preview image.

14. A method of outputting an avatar image of an electronic device, the method comprising:
obtaining, using a camera, a preview image including an object;
tracking the object in the obtained preview image;
determining whether or not the object includes a face of a user;
producing the avatar image including an avatar which has features corresponding to the user's face and/or which simulates a motion of the object based on the tracking of the object;
synthesizing the avatar image with at least a portion of the object in the preview image;
outputting, through a display, the preview image such that the avatar image replaces the user's face in the preview image; and
select between providing a facial avatar and providing a full body avatar according to whether a first portion or a second portion of a body of the user is a predetermined size relative to the object in the preview image.

15. The method of claim 14 , further comprising outputting a user interface for changing at least one aspect of the avatar image, and
wherein producing the avatar image includes selecting avatar features reflecting the features of the user face for inclusion in the avatar image, such that the produced avatar image resembles the user's face.

16. The method of claim 15 , further comprising:
outputting a plurality of selection items for changing various attribute information of the avatar; and
in response to detecting a selection of one of the selection items from among the plurality of selection items, modifying a visual appearance of the avatar by incorporating a selected characteristic into avatar,
wherein the plurality of selection items includes sequential display of a first menu, a second menu and a third menu, and
wherein the first menu including selectable items to indicate an applicable sex of the synthesized object, the second menu including selectable items representing applicable facial features of the synthesized object, and the third menu including selectable items to indicate applicable costume of the synthesized object.

17. The method of claim 14 , further comprising:
tracking distinguished features of the object in the obtained preview image; and
producing the avatar image based on the distinguished features,
wherein the tracked distinguished features correspond to a face contour, a pupil, a mouth, a hand, a finger, and/or a body included in the object.

18. The method of claim 14 , wherein synthesizing of the object with the avatar image further comprises:
synthesizing at least a portion of the object in the preview image with the avatar image such that the avatar image is displayed together with an image corresponding to the object of the previ

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Wooyong
• Baik Seunghyup
• Park Chanmin
• Kang Hyejin
• Lee Jungeun

Classifications

• H04N23/80

  Camera processing pipelines; Components thereof · CPC title

• H04N23/95

  Computational photography systems, e.g. light-field imaging systems · CPC title

• G06V40/168

  Feature extraction; Face representation · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title