System and method for positioning a 3D eyeglasses model

Patent metadata

Field	Value
Publication number	US-12183035-B1
Application number	US-202217687268-A
Country	US
Kind code	B1
Filing date	Mar 4, 2022
Priority date	Mar 8, 2021
Publication date	Dec 31, 2024
Grant date	Dec 31, 2024

Abstract

In some embodiments, a computer-implemented method includes obtaining a without-eyeglasses face scan of a subject, the without-eyeglasses face scan being a three-dimensional (3D) model of a face of the subject without eyeglasses; obtaining a with-eyeglasses face scan of the subject, the with-eyeglasses face scan being a 3D model of the subject with eyeglasses; and using the without-eyeglasses face scan and the with-eyeglasses face scan to place a 3D eyeglasses model on a face model of the subject. In some embodiments of the computer-implemented method, the 3D eyeglasses model is placed on the face model of the subject using frame placement information generated using the without-eyeglasses face scan and the with-eyeglasses face scan.

First claim

What is claimed is:

1. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to:
generate eye landmarks from a without-eyeglasses face scan of a subject, the without-eyeglasses face scan being a three-dimensional (3D) model of a face of the subject without eyeglasses;
generate a bounding volume of a frontal frame of eyeglasses using the eye landmarks;
generate frame points from the without-eyeglasses face scan and a with-eyeglasses face scan inside the bounding volume; and
use the frame points and a frontal image of the without-eyeglasses face scan to generate frame placement information associated with a positioning of a 3D eyeglasses model on a face model.

2. The non-transitory computer-readable storage medium of claim 1 , wherein:
the frame placement information includes a first portion of the frame placement information and a second portion of the frame placement information.

3. The non-transitory computer-readable storage medium of claim 2 , wherein:
the first portion of the frame placement information includes an X rotation of a frontal frame of an eyeglasses frame, a Y rotation of the frontal frame of the eyeglasses frame, a Z distance from the frontal frame to an origin on the face model, and the second portion of the frame placement information includes an X translation of the frontal frame of the eyeglasses frame, a Y translation of the frontal frame of the eyeglasses frame, and a Z rotation of the frontal frame of the eyeglasses frame.

4. The non-transitory computer-readable storage medium of claim 2 , wherein: the first portion of the frame placement information is generated using the frame points and a 3D plane.

5. The non-transitory computer-readable storage medium of claim 4 , wherein:
the frame points are generated using the bounding volume.

6. The non-transitory computer-readable storage medium of claim 4 , wherein:
the first portion of the frame placement information is generated by fitting the 3D plane to the frame points.

7. The non-transitory computer-readable storage medium of claim 2 , wherein:
the second portion of the frame placement information is generated using a frontal image of the without-eyeglasses face scan.

8. The non-transitory computer-readable storage medium of claim 3 , wherein:
the bounding volume is used to generate the X translation of the frontal frame of the eyeglasses frame, the Y translation of the frontal frame of the eyeglasses frame, and the Z rotation of the frontal frame of the eyeglasses frame.

9. A device, comprising:
a processor; and
a memory in communication with the processor for storing instructions, which when executed by the processor causes the device to:
generate eye landmarks from a without-eyeglasses face scan of a subject;
generate a bounding volume of a frontal frame of eyeglasses using the eye landmarks;
generate frame points from the without-eyeglasses face scan and a with-eyeglasses face scan inside the bounding volume; and
use the frame points and a frontal image of the without-eyeglasses face scan to generate frame placement information associated with a positioning of a 3D model of a pair of eyeglasses on a face model.

10. The device of claim 9 , wherein:
the frame placement information includes a first portion of the frame placement information and a second portion of the frame placement information.

11. The device of claim 10 , wherein:
the first portion of the frame placement information includes an X rotation of a frontal frame of an eyeglasses frame, a Y rotation of the frontal frame of the eyeglasses frame, a Z distance from the frontal frame to an origin on the face model, and the second portion of the frame placement information includes an X translation of the frontal frame of the eyeglasses frame, a Y translation of the frontal frame of the eyeglasses frame, and a Z rotation of the frontal frame of the eyeglasses frame.

12. The device of claim 10 , wherein: the first portion of the frame placement information is generated using the frame points and a 3D plane.

13. The device of claim 12 , wherein:
the frame points are generated using the bounding volume.

14. The device of claim 12 , wherein:
in order to generate the first portion of the frame placement information, the 3D plane is fit to the frame points.

15. The device of claim 10 , wherein:
the second portion of the frame placement information is generated using a frontal image of the without-eyeglasses face scan.

16. The device of claim 11 , wherein:
a bounding box is used to generate the X translation of the frontal frame of the eyeglasses frame, the Y translation of the frontal frame of the eyeglasses frame, and the Z rotation of the frontal frame of the eyeglasses frame.

17. A method, comprising:
generating eye landmarks from a without-eyeglasses face scan of a subject;
generating a bounding volume of a frontal frame of eyeglasses using the eye landmarks;
generating frame points from the without-eyeglasses face scan and a with-eyeglasses face scan inside the bounding volume; and
using the frame points and a frontal image of the without-eyeglasses face scan to generate frame placement information associated with a positioning of a 3D eyeglasses model on a face model.

18. The method of claim 17 , wherein:
the frame points are used to recover an X rotation of a frontal frame of an eyeglasses frame, a Y rotation of the frontal frame of the eyeglasses frame, and a Z distance of the frontal frame of the eyeglasses frame.

19. The method of claim 18 , wherein:
a plane is fit to the frame points in order to generate the X rotation of the frontal frame of the eyeglasses frame, the Y rotation of the frontal frame of the eyeglasses frame, and the Z distance of the frontal frame of the eyeglasses frame.

20. The method of claim 17 , wherein:
the frontal image of the without-eyeglasses face scan is used to recover an X translation of the frontal frame of the eyeglasses frame, a Y translation of the frontal frame of the eyeglasses frame, and a Z rotation of the frontal frame of the eyeglasses frame.

Assignees

• Meta Platforms Inc

Inventors

• Chen Shenchang Eric

Classifications

• G06V40/171

  Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T2210/16

  Cloth · CPC title

• G06T2210/12

  Bounding box · CPC title