Augmented reality enhancements for intraoral scanning

Patent metadata

Field	Value
Publication number	US-10695150-B2
Application number	US-201715841200-A
Country	US
Kind code	B2
Filing date	Dec 13, 2017
Priority date	Dec 16, 2016
Publication date	Jun 30, 2020
Grant date	Jun 30, 2020

Abstract

A system comprises a scanner, an augmented reality (AR) display and a computing device. The scanner generates intraoral images of a dental arch and the AR display generates additional image data representative of a view from a wearer of the AR display. The computing device receives the intraoral images from the intraoral scanner, generates a virtual three-dimensional model of at least a portion of the dental arch from the intraoral images, receives the additional image data from the AR display, determines, from the additional image data, a region of the view that is outside of the dental arch, generates a visual overlay comprising the virtual three-dimensional model, and sends the visual overlay to the AR display. The AR display displays the visual overlay such that the virtual three-dimensional model is shown in the region of the view that is outside of the dental arch.

First claim

What is claimed is:

1. A system comprising:
an intraoral scanner to generate a plurality of intraoral images of a dental arch of a patient during an intraoral scanning procedure;
an augmented reality (AR) display comprising an image capture device and one or more lenses, the image capture device of the AR display to generate additional image data representative of a field of view of a wearer of the AR display; and
a computing device operatively coupled to the intraoral scanner and the AR display, the computing device to:
receive the plurality of intraoral images from the intraoral scanner;
generate a virtual three-dimensional model of at least a portion of the dental arch from the plurality of intraoral images during the intraoral scanning procedure;
receive the additional image data from the AR display;
determine, from the additional image data, a first region of the field of view that includes the dental arch and a second region of the field of view that does not include the dental arch;
generate a visual overlay of the virtual three-dimensional model; and
send the visual overlay of the virtual three-dimensional model to the AR display, wherein the AR display is to superimpose the visual overlay of the virtual three-dimensional model over a real-world environment in the field of view of the wearer that the wearer sees through the one or more lenses such that the visual overlay of the virtual three-dimensional model is superimposed over the real-world environment in the second region of the field of view and the visual overlay of the virtual three-dimensional model is not superimposed over the real-world environment in the first region of the field of view that includes the dental arch.

2. The system of claim 1 , wherein the computing device is further to:
determine an area of the dental arch that has been scanned based on the plurality of intraoral images; and
generate an additional visual overlay comprising at least one of a first visual indication of the area that has been scanned or a second visual indication of an additional area of the dental arch that has not been scanned.

3. The system of claim 2 , wherein the computing device is further to:
perform a diagnosis of the area of the dental arch that has been scanned; and
add an indication of the diagnosis to the additional visual overlay.

4. The system of claim 1 , wherein the computing device is further to:
determine a stage of the intraoral scanning procedure;
determine a plurality of menu options for the stage of the intraoral scan procedure; and
add a display of the plurality of menu options to the visual overlay.

5. The system of claim 1 , wherein the computing device is further to:
determine a location of the intraoral scanner and of an area of the dental arch proximate to the intraoral scanner in the additional image data;
generate a zoomed in view of the location of the intraoral scanner and of the area of the dental arch proximate to the intraoral scanner; and
add the zoomed in view to the visual overlay, wherein the AR display is to display the visual overlay such that the zoomed in view is shown in the second region of the field of view.

6. The system of claim 1 , further comprising:
a virtual reality (VR) display to be worn by the patient;
wherein the computing device is further to send at least one of the additional image data or the visual overlay to the VR display.

7. The system of claim 1 , wherein the plurality of intraoral images comprise images of a dental tool proximate to an area of the dental arch, and wherein the computing device is further to:
generate a zoomed in view of the dental tool and of the area of the dental arch proximate to the dental tool from the plurality of intraoral images; and
add the zoomed in view to the visual overlay, wherein the AR display is to display the visual overlay such that the zoomed in view is shown in the second region of the field of view.

8. A method comprising:
receiving a plurality of intraoral images of a patient from an intraoral scanner during an intraoral scanning procedure;
generating, by a computing device, a virtual three-dimensional model of at least a portion of a dental arch from the plurality of intraoral images during the intraoral scanning procedure;
receiving additional image data from an augmented reality (AR) display comprising one or more lenses, wherein the additional image data is representative of a field of view of a wearer of the AR display;
determining, from the additional image data, a first region of the field of view that includes the dental arch and a second region of the field of view that does not include the dental arch;
generating, by the computing device, a visual overlay of the virtual three-dimensional model; and
sending the visual overlay of the virtual three-dimensional model to the AR display, wherein the AR display is to superimpose the visual overlay of the virtual three-dimensional model over a real-world environment in the field of view of the wearer that the wearer sees through the one or more lenses such that the visual overlay of the virtual three-dimensional model is superimposed over the real-world environment in the second region of the field of view and the visual overlay of the virtual three-dimensional model is not superimposed over the real-world environment in the first region of the field of view that includes the dental arch.

9. The method of claim 8 , further comprising:
determining an area of the dental arch that has been scanned based on the plurality of intraoral images; and
generating an additional visual overlay comprising at least one of a first visual indication of the area that has been scanned or a second visual indication of an additional area of the dental arch that has not been scanned.

10. The method of claim 9 , further comprising:
performing a diagnosis of the area of the dental arch that has been scanned; and
adding an indication of the diagnosis to the additional visual overlay.

11. The method of claim 8 , further comprising:
determining a stage of the intraoral scanning procedure;
determining a plurality of menu options for the stage of the intraoral scan procedure; and
adding a display of the plurality of menu options to the visual overlay.

12. The method of claim 8 , further comprising:
determining a location of the intraoral scanner and of an area of the dental arch proximate to the intraoral scanner in the additional image data;
generating a zoomed in view of the location of the intraoral scanner and of the area of the dental arch proximate to the intraoral scanner; and
adding the zoomed in view to the visual overlay, wherein the AR display is to display the visual overlay such that the zoomed in view is shown in the second region of the field of view.

13. The method of claim 8 , further comprising:
sending at least one of the additional image data or the visual overlay to a virtual reality (VR) display worn by the patient.

14. The method of claim 8 , wherein the plurality of intraoral images comprise images of a dental tool proximate to an area of the dental arch, the method further comprising:
generating a zoomed in view of the dental tool and of the area of the dental arch proximate to the dental tool from the plurality of intraoral images; and
adding the zoomed in view to the visual overlay, wherein the AR display is to display the visual overlay such that the zoomed in view is shown in the second region of the field of view.

15. A non-transitory computer readable storage medium comprising instructions that, when executed by a processing device, cause the

Assignees

• Align Technology Inc

Inventors

• Kopelman Avi
• Meyer Eric Paul
• Zeiri Elad
• Ashkenazi Amir
• Ganot Ron
• Dey Partha
• Ozerov Sergei
• Mizrahi Ran
• Fomin Ilya
• Nolen Sean M
• Valiev Sergey
• Fridman Edi
• Gagarin Sergey

Classifications

• G06V10/758

  Involving statistics of pixels or of feature values, e.g. histogram matching · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/09

  Supervised learning · CPC title

• A61B1/000096

  using artificial intelligence · CPC title

• A61B1/000094

  extracting biological structures · CPC title