Rendering of post treatment smile image

Patent metadata

Field	Value
Publication number	US-12133783-B2
Application number	US-202318142545-A
Country	US
Kind code	B2
Filing date	May 2, 2023
Priority date	May 21, 2018
Publication date	Nov 5, 2024
Grant date	Nov 5, 2024

Abstract

Embodiments may include: receiving a facial image comprising a 2D depiction of a person's teeth at a first arrangement; receiving a tooth model comprising a 3D depiction of the person's teeth at a second arrangement; obtaining color information from the facial image, wherein the color information represents a color scheme of the person's teeth at the first arrangement; and generating a photo-realistic two-dimensional depiction of the person's teeth at the second arrangement using the 3D depiction of the person's teeth at the second arrangement and the color information obtained from the facial image.

First claim

What is claimed is:

1. A system comprising:
a memory; and
one or more processors operatively connected to the memory, the one or more processors to:
receive a facial image comprising a two-dimensional (2D) depiction of a person's teeth at a first arrangement;
receive a tooth model comprising a three-dimensional (3D) depiction of the person's teeth at a second arrangement;
obtain color information from the facial image, wherein the color information represents a color scheme of the person's teeth at the first arrangement;
code one or more channels for a plurality of pixels of a 2D rendering of the tooth model with structural attributes of the 3D depiction; and
generate a photo-realistic 2D depiction of the person's teeth at the second arrangement using a result of coding the one or more channels for the plurality of pixels of the 2D rendering and the color information obtained from the facial image.

2. The system of claim 1 , wherein the system comprises a mobile phone, the system further comprising:
a camera to capture the facial image;
wherein the one or more processors are to receive the facial image from the camera or the memory.

3. The system of claim 1 , wherein the facial image is received from a mobile phone.

4. The system of claim 1 , wherein to obtain the color information the one or more processors are to blur the 2D depiction of the person's teeth at the first arrangement.

5. The system of claim 1 , wherein to obtain the color information the one or more processors are to obtain one or more average values of pixels in the 2D depiction of the person's teeth at the first arrangement.

6. The system of claim 1 , wherein the tooth model comprises a 3D model of the person's teeth derived from an intraoral scan of the person's teeth.

7. The system of claim 1 , wherein the one or more processors are further to:
receive an intraoral scan of the person's teeth at the first arrangement;
segment the intraoral scan into a 3D model of the person's teeth at the first arrangement;
identify the second arrangement using a treatment plan; and
modify one or more attributes of the 3D model of the person's teeth at the first arrangement to create the tooth model comprising the 3D depiction of the person's teeth at the second arrangement.

8. The system of claim 1 , further comprising:
a display, wherein the one or more processors are further to output the photo-realistic 2D depiction of the person's teeth to the display.

9. The system of claim 1 , further comprising:
a network interface, wherein the system is to transmit the photo-realistic 2D depiction of the person's teeth to a remote device via the network interface, and wherein the remote device is to output the photo-realistic 2D depiction of the person's teeth to a display.

10. The system of claim 1 , wherein the person's teeth at the second arrangement corresponds to an estimated arrangement of the person's teeth after treatment according to a treatment plan, and wherein the one or more processors are further to:
receive approval of the treatment plan based at least in part on the photo-realistic 2D depiction of the person's teeth at the second arrangement; and
implement the treatment plan.

11. The system of claim 1 , wherein the one or more processors are further to:
design one or more dental appliances to be applied to the person's teeth to effect a treatment plan; and
cause a fabrication machine to fabricate the one or more dental appliances.

12. The system of claim 1 , wherein the tooth model comprises a 3D model of the person's teeth derived from a plurality of 2D images of the person's teeth.

13. The system of claim 1 , wherein the structural attributes comprise an identity of a tooth or a gingiva corresponding to each of the pixels.

14. The system of claim 1 , wherein the one or more channels comprise at least one of:
a first channel that encodes angles of normals for locations on the tooth model;
a second channel that encodes luminosity with respect to the normals for the locations on the tooth model;
a third channel that encodes depth information for the locations on the tooth model;
a fourth channel that encodes brightness information for the locations on the tooth model;
a fifth channel that encodes a dental structure type for the locations on the tooth model; or
a sixth channel that encodes an angle with respect to a facial plane for the locations on the tooth model.

15. The system of claim 1 , wherein the one or more processors are further to:
whiten the person's teeth in the photo-realistic 2D depiction of the person's teeth.

16. The system of claim 15 , wherein to whiten the person's teeth the one or more processors are further to:
convert the facial image from a first color space to a LAB color space; and
decrease a value for a B channel in the LAB color space for pixels of the photo-realistic 2D depiction of the person's teeth.

17. The system of claim 1 , wherein the one or more processors are further to:
brighten the person's teeth in the photo-realistic 2D depiction of the person's teeth.

18. The system of claim 17 , wherein to brighten the person's teeth the one or more processors are further to:
convert the facial image from a first color space to a LAB color space; and
increase a value for a L channel in the LAB color space for pixels of the photo-realistic 2D depiction of the person's teeth.

19. The system of claim 1 , wherein the one or more processors are further to:
add specular highlights to the photo-realistic 2D depiction of the person's teeth.

20. The system of claim 19 , wherein to add the specular highlights the one or more processors are further to:
identify locations in the photo-realistic 2D depiction that represent teeth;
identify locations in the photo-realistic 2D depiction having a brightness that exceeds a brightness threshold;
identify normals at locations in the photo-realistic 2D depiction; and
determine the specular highlights for the teeth based on the locations in the photo-realistic 2D depiction, the locations in the photo-realistic 2D depiction having a brightness that exceeds the brightness threshold, and the normals at the locations in the photo-realistic 2D depiction.

21. The system of claim 1 , wherein the color information comprises values for a first plurality of color channels, and wherein the structural attributes of the 3D depiction are coded into a second plurality of channels.

22. The system of claim 21 , wherein the first plurality of color channels comprises a first three color channels, and wherein the second plurality of channels comprises a second three channels.

23. A method comprising:
receiving a facial image comprising a two-dimensional (2D) depiction of a person's teeth at a first arrangement;
receiving a tooth model comprising a three-dimensional (3D) depiction of the person's teeth at a second arrangement;
obtaining color information from the facial image, wherein the color information represents a color scheme of the person's teeth at the first arrangement;
coding one or more channels for a plurality of pixels of a 2D rendering of the tooth model with structural attributes of the 3D depiction; and
generating a photo-realistic 2D depiction of the person's teeth at the second arrangement using a result of coding the one or more channels for the plurality of pixels of the 2D rendering and the color information obtained from the facial image.

24. The meth

Assignees

• Align Technology Inc

Inventors

• Chekh Dmitry Yurievich
• Blanco Samuel
• Miryaha Vladislav Andreevich
• Vysokanov Boris Aleksandrovich
• Brown Chad Clayton
• Cramer Christopher E

Classifications

• G06T5/70

  Denoising; Smoothing · CPC title

• H04N1/3871

  the composed originals being of different kinds, e.g. low- and high-resolution originals · CPC title

• G06T15/83

  Phong shading · CPC title

• G06T2219/2012

  Colour editing, changing, or manipulating; Use of colour codes · CPC title

• H04N1/646

  Transmitting or storing colour television type signals, e.g. PAL, Lab; Their conversion into additive or subtractive colour signals or vice versa therefor (H04N1/642, H04N1/644 take precedence) · CPC title