Method and electronic device for generating three-dimensional (3D) model

Patent metadata

Field	Value
Publication number	US-12499617-B2
Application number	US-202318454402-A
Country	US
Kind code	B2
Filing date	Aug 23, 2023
Priority date	Oct 7, 2022
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

A method of generating a three-dimensional (3D) model and a device for generating a 3D model are provided. The method includes generating a base 3D model of an object, determining a first target 3D model similar to the base 3D model from a 3D model database storing a plurality of 3D models, generating a first synthesized 3D model, determining a first similarity between the first synthesized 3D model and the at least one object image, determining a component having a low similarity between the first synthesized 3D model and the at least one object image, determining, as a second target 3D model, a 3D model having a high similarity to the first synthesized 3D model with respect to the component having the low similarity, from the 3D model database, and generating a second synthesized 3D model.

First claim

What is claimed is:

1 . A method of generating a three-dimensional (3D) model, the method comprising:
generating a base 3D model of an object, based on at least one object image; determining a first target 3D model similar to the base 3D model from a 3D model database storing a plurality of 3D models; generating a first synthesized 3D model, based on the base 3D model and the first target 3D model; determining a first similarity between the first synthesized 3D model and the at least one object image; determining a component having a low similarity between the first synthesized 3D model and the at least one object image, based on the first similarity being less than or equal to a threshold value; determining, as a second target 3D model, a 3D model having a high similarity to the first synthesized 3D model with respect to the component having the low similarity, from the 3D model database; and generating a second synthesized 3D model, based on the first synthesized 3D model and the second target 3D model.

2 . The method of claim 1 , wherein the generating of the base 3D model comprises:
obtaining depth information of the at least one object image by using the at least one object image; and generating the base 3D model including 3D location information about at least a portion of the object by using the at least one object image and the obtained depth information.

3 . The method of claim 1 , wherein the determining of the first target 3D model comprises:
determining a second similarity between the base 3D model and the plurality of 3D models; and determining, as the first target 3D model, a 3D model having a high second similarity among the plurality of 3D models.

4 . The method of claim 1 , wherein the generating of the first synthesized 3D model comprises:
performing deformation on the first target 3D model to correspond to the base 3D model; and merging the base 3D model and deformed first target 3D model.

5 . The method of claim 1 , further comprising:
classifying the base 3D model into a plurality of components of the object.

6 . The method of claim 5 , wherein the determining of the component having the low similarity comprises:
determining a third similarity between the first synthesized 3D model and the at least one object image with respect to each of the plurality of classified components; and determining, as the component having the low similarity, a component having the third similarity less than a certain value among the plurality of components.

7 . The method of claim 1 , wherein the determining of the first similarity comprises:
rendering the first synthesized 3D model; modifying a pose of the rendered first synthesized 3D model to correspond to the at least one object image; and determining the first similarity between the modified rendered first synthesized 3D model and the at least one object image.

8 . The method of claim 1 , further comprising:
identifying a type of the object, based on the at least one object image, wherein the 3D model database comprises 3D models related to the type of the object.

9 . The method of claim 1 , wherein the generating of the first synthesized 3D model comprises:
generating information about a location of the first synthesized 3D model, based on information about a location of the base 3D model and information about a location of the first target 3D model; and generating information about a color of the first synthesized 3D model, based on information about a color of the base 3D model.

10 . The method of claim 1 , wherein the first synthesized 3D model comprises at least one of a point cloud model, a polygon mesh model, a triangle mesh model, a non-uniform rational B-spline (NURBS) model, or a sculpting model.

11 . An electronic device for generating a three-dimensional (3D) model, the electronic device comprising:
at least one processor; and a memory storing at least one instruction, wherein the at least one processor is configured to execute the at least one instruction to:
generate a base 3D model of an object, based on at least one object image,
determine a first target 3D model similar to the base 3D model from a 3D model database storing a plurality of 3D models,
generate a first synthesized 3D model, based on the base 3D model and the first target 3D model,
determine a first similarity between the first synthesized 3D model and the at least one object image,
determine a component having a low similarity between the first synthesized 3D model and the at least one object image, based on the first similarity being less than or equal to a threshold value,
determine, as a second target 3D model, a 3D model having a high similarity to the first synthesized 3D model with respect to the component having the low similarity, from the 3D model database, and
generate a second synthesized 3D model based on the first synthesized 3D model and the second target 3D model.

12 . The electronic device of claim 11 , wherein the at least one processor is further configured to:
execute the at least one instruction to obtain depth information of the at least one object image by using the at least one object image, and generate the base 3D model including 3D location information about at least a portion of the object by using the at least one object image and the obtained depth information.

13 . The electronic device of claim 11 , wherein the at least one processor is further configured to:
execute the at least one instruction to determine a second similarity between the at least one object image and the plurality of 3D models, and determine, as the first target 3D model, a 3D model having a high second similarity among the plurality of 3D models.

14 . The electronic device of claim 11 , wherein the at least one processor is further configured to:
execute the at least one instruction to perform deformation on the first target 3D model to correspond to the base 3D model, and merge the base 3D model and the deformed first target 3D model.

15 . The electronic device of claim 11 , wherein the at least one processor is further configured to execute the at least one instruction to classify the base 3D model into a plurality of components of the object.

16 . The electronic device of claim 15 , wherein the at least one processor is further configured to:
execute the at least one instruction to determine a third similarity between the first synthesized 3D model and the at least one object image with respect to each of the plurality of classified components, and determine, as the component having the low similarity, a component having the third similarity less than a certain value among the plurality of components.

17 . The electronic device of claim 11 , wherein the at least one processor is further configured to:
execute the at least one instruction to render the first synthesized 3D model, modify a pose of the rendered first synthesized 3D model to correspond to the at least one object image, and determine the first similarity between the modified rendered first synthesized 3D model and the at least one object image.

18 . The electronic device of claim 11 ,
wherein the at least one processor is further configured to execute the at least one instruction to identify a type of the object based on the at least one object image, and wherein the 3D model database comprises 3D models related to the type of the object.

19 . The electronic device of claim 11 , wherein the at least one processor is further

Assignees

• Samsung Electronics Co Ltd

Inventors

• Shin Jaewook
• Kim Dongchan
• Byun Dongnam
• Hwang Jinyoung

Classifications

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2219/2004

  Aligning objects, relative positioning of parts · CPC title

• G06T2219/2021

  Shape modification · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06T2219/2016

  Rotation, translation, scaling · CPC title