Excess material removal using machine learning

Patent metadata

Field	Value
Publication number	US-11238586-B2
Application number	US-202016865162-A
Country	US
Kind code	B2
Filing date	May 1, 2020
Priority date	May 2, 2019
Publication date	Feb 1, 2022
Grant date	Feb 1, 2022

Abstract

A method includes processing an input comprising data from an intraoral image using a trained machine learning model that has been trained to classify regions of dental sites, wherein the trained machine learning model outputs a probability map comprising, for each pixel in the intraoral image, a first probability that the pixel belongs to a first dental class and a second probability that the pixel belongs to a second dental class, wherein the first dental class represents excess material, the excess material comprising material other than teeth or gums. The method further includes determining, based on the probability map, one or more pixels in the intraoral image that are classified as excess material. The method further includes hiding or removing from the intraoral image data for the one or more pixels that are classified as excess material.

First claim

What is claimed is:

1. A method comprising:
receiving an intraoral image of a dental site, the intraoral image comprising a height map;
processing an input comprising data from the intraoral image using a trained machine learning model that has been trained to classify regions of dental sites, wherein the trained machine learning model outputs a probability map comprising, for each pixel in the intraoral image, a first probability that the pixel belongs to a first dental class and a second probability that the pixel belongs to a second dental class, wherein the first dental class represents excess material, the excess material comprising material other than teeth or gums;
determining, based on the probability map, one or more pixels in the intraoral image that are classified as excess material; and
hiding or removing, from the intraoral image, data for the one or more pixels that are classified as excess material.

2. The method of claim 1 , further comprising:
receiving a plurality of additional intraoral images of the dental site;
processing a plurality of additional inputs using the trained machine learning model, wherein each additional input comprises data from one of the plurality of additional intraoral images, wherein for each additional input the trained machine learning model outputs an additional probability map comprising, for each pixel in an additional intraoral image associated with the additional input, an additional first probability that the pixel belongs to the first dental class and an additional second probability that the pixel belongs to the second dental class;
for each additional intraoral image, determining zero or more pixels in the additional intraoral image that are classified as excess material based on the additional probability map associated with the additional intraoral image;
for each additional intraoral image comprising one or more pixels that are classified as excess material, hiding or removing, from the additional intraoral image, data for the one or more pixels in the additional intraoral image that are classified as excess material; and
generating a three-dimensional model of the dental site from the intraoral image and the plurality of additional intraoral images.

3. The method of claim 1 , wherein the intraoral image is received from an intraoral scanner during an intraoral scanning session, and wherein the processing of the intraoral image is performed during the intraoral scanning session while another intraoral image is generated by the intraoral scanner.

4. The method of claim 1 , wherein the intraoral image is a blended image comprising data from a sequence of raw intraoral images, the method further comprising:
combining the data from the sequence of raw intraoral images to generate the blended image.

5. The method of claim 1 , further comprising:
receiving an additional image of the dental site that was generated prior to the intraoral image, wherein the input comprises the data from the intraoral image and additional data from the additional image.

6. The method of claim 5 , further comprising:
generating a three-dimensional model of the dental site from a plurality of previously received intraoral images;
determining a plane associated with the intraoral image; and
projecting the three-dimensional model of the dental site onto the plane to generate the additional image.

7. The method of claim 1 , further comprising:
receiving a color image of the dental site that is associated with the intraoral image of the dental site, wherein the input comprises the data from the intraoral image and additional data from the color image.

8. The method of claim 7 , further comprising:
determining an alignment between the color image and the intraoral image.

9. The method of claim 1 , wherein the trained machine learning model comprises a recurrent neural network, and wherein the input further comprises a previous output of the recurrent neural network associated with a previous intraoral image of the dental site.

10. The method of claim 1 , further comprising:
receiving an additional image of the dental site generated under specific lighting conditions, wherein the input comprises the data from the intraoral image and additional data from the additional image, and wherein the specific lighting conditions comprise infrared illumination or ultraviolet illumination.

11. The method of claim 1 , wherein each pixel in the intraoral image comprises a first value representing height and a second value representing intensity, and wherein the data from the intraoral image that is included in the input comprises the first value and the second value for each pixel.

12. The method of claim 1 , wherein the probability map comprises, for each pixel in the intraoral image, the first probability that the pixel belongs to the first dental class, the second probability that the pixel belongs to the second dental class, and a third probability that the pixel belongs to a third dental class, wherein the second dental class represents teeth, and wherein the third dental class represents gums.

13. The method of claim 12 , wherein:
the probability map comprises an RGB image;
R values of the RGB image are associated with a first one of the first dental class, the second dental class or the third dental class;
G values in the RGB image are associated with a second one of the first dental class, the second dental class or the third dental class; and
B values in the RGB image are associated with a third one of the first dental class, the second dental class or the third dental class.

14. The method of claim 1 , wherein determining that a pixel is classified as excess material comprises:
determining that the first probability that the pixel belongs to the first dental class that represents excess material exceeds a probability threshold.

15. The method of claim 1 , wherein determining that a pixel is classified as excess material comprises:
determining that the first probability that the pixel belongs to the first dental class that represents excess material exceeds the second probability that the pixel belongs to the second dental class.

16. The method of claim 1 , wherein the probability map comprises, for each pixel in the intraoral image, the first probability that the pixel belongs to the first dental class, the second probability that the pixel belongs to the second dental class, a third probability that the pixel belongs to a third dental class, and a fourth probability that the pixel belongs to a fourth dental class, wherein the first dental class represents excess material not adjacent to teeth, wherein the second dental class represents teeth, wherein the third dental class represents gums, and wherein the fourth dental class represents excess material adjacent to teeth.

17. The method of claim 1 , wherein the probability map comprises, for each pixel in the intraoral image, the first probability that the pixel belongs to the first dental class, the second probability that the pixel belongs to the second dental class, a third probability that the pixel belongs to a third dental class, and a fourth probability that the pixel belongs to a fourth dental class, wherein the second dental class represents teeth, wherein the third dental class represents gums, and wherein the fourth dental class represents one of an upper palate, a gingival line, a scan body, a finger, or a preparation tooth.

18. The method of claim 1 , further comprising:
generating a virtual three-dimensional (3D) model of the dental site using the

Assignees

• Align Technology Inc

Inventors

• Minchenkov Mikhail
• Katz Ran
• Agniashvili Pavel
• Brown Chad Clayton
• Coslovsky Jonathan

Classifications

• G06V20/653

  by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06T7/11 Primary

  Region-based segmentation · CPC title

• G06T7/0012 Primary

  Biomedical image inspection · CPC title