Photograph-based assessment of dental treatments and procedures

Patent metadata

Field	Value
Publication number	US-10248883-B2
Application number	US-201514831548-A
Country	US
Kind code	B2
Filing date	Aug 20, 2015
Priority date	Aug 20, 2015
Publication date	Apr 2, 2019
Grant date	Apr 2, 2019

Abstract

The current document is directed to methods and systems for monitoring a dental patient's progress during a course of treatment. A three-dimensional model of the expected positions of the patient's teeth can be projected, in time, from a three-dimensional model of the patient's teeth prepared prior to beginning the treatment. A digital camera is used to take one or more two-dimensional photographs of the patient's teeth, which are input to a monitoring system. The monitoring system determines virtual-camera parameters for each two-dimensional input image with respect to the time-projected three-dimensional model, uses the determined virtual-camera parameters to generate two-dimensional images from the three-dimensional model, and then compares each input photograph to the corresponding generated two-dimensional image in order to determine how closely the three-dimensional arrangement of the patient's teeth corresponds to the time-projected three-dimensional arrangement.

First claim

The invention claimed is:

1. A dental-treatment monitoring system comprising:
one or more processors;
one or more electronic memories that store instructions and data;
one or more mass-storage devices that store encoded images and patient information;
a communications subsystem through which the dental-treatment monitoring system receives images and information from remote computer systems; and
computer instructions, encoded in one or more of the one or more electronic memories, that control the dental-treatment monitoring system to store, in one or more data-storage devices selected from among the one or more electronic memories and one or more mass-storage devices, a three-dimensional model of a patient's teeth, receive, through the communications subsystem, one or more two-dimensional digital photographs of the patient's teeth taken at a time t during the course of a dental treatment, project, in time, the three-dimensional model of the patient's teeth to the time t to generate and store a time-projected three-dimensional model of the patient's teeth representing an expected configuration of the patient's teeth, compare one or more of the one or more two-dimensional digital photographs to corresponding two-dimensional digital images generated from the time-projected three-dimensional model of the patient's teeth to generate one or more comparison values and generate, from the received one or more two-dimensional digital photographs, a set of one or more processed images with associated metadata for analysis, determine, for each image in the set of one or more processed images, a set of virtual-camera parameters that describe the position, and orientation for a virtual camera that produces a generated image from the time-projected three-dimensional model of the patient's teeth equivalent to the image in the set of one or more processed images, use, for each image in the set of one or more processed images, the standard type of view and additional metadata to generate an initial set of virtual-camera parameters for the image in the set of one or more processed images, use the standard type of view and additional metadata to determine a value for an iteration variable; carry out a number of optimization iterations equal to the value of the iteration variable; and finally refine the virtual-camera parameters for the image in the set of one or more processed images, generate, for each image in the set of one or more processed images, a generated image corresponding to the image from the time-projected three-dimensional model of the patient's teeth and the virtual-camera parameters determined for the image in the set of one or more processed images, and compare each image in the set of one or more processed images with the corresponding generated image to generate the one or more comparison values for the image in the set of one or more processed images, determine, from the one or more comparison values, whether or not a configuration of the patient's teeth is within a threshold level of correspondence to the expected configuration of the patient's teeth, and store an indication of the determination in one of the one or more electronic memories.

2. The dental-treatment monitoring system of claim 1 wherein the received two-dimensional digital photographs are associated with a patient identifier; and wherein each of the received two-dimensional digital photographs are associated with metadata that includes one or more of: one or more text labels for the two-dimensional image, including an indication of a standard type of view represented by the image, and characterizations and parameters for the digitally encoded image, including an image size, date and time information, an indication of a camera model and make, an indication of a camera orientation, an indication of an aperture, an indication of a shutter speed, an indication of a focal length, an indication of a metering mode, and an indication of a speed.

3. The dental-treatment monitoring system of claim 2 further comprises verifying and filtering, by the dental-treatment monitoring system, the patient ID and the received two-dimensional digital photographs and associated metadata to produce a set of one or more processed images for analysis by: accessing stored information to verify that the patient ID corresponds to a patient record; for each input image, determining that the image corresponds to the standard type of view indicated by the one or more text labels, applying filters and contrast adjustment to enhance the image for subsequent processing, when the image is the first image of a standard type of view, adding the image and associated metadata to the set of one or more processed images, and when the image has the same standard type of view as an image already added to the set of one or more processed images, replacing the image already added to the set of one or more processed images with the image and associated metadata; and determining whether the set of one or more processed images can be used for dental-treatment-progress analysis.

4. The dental-treatment monitoring system of claim 1 wherein each optimization iteration includes: a first adjustment of the current virtual-camera parameters for the image in the set of one or more processed images; a second adjustment of the current virtual-camera parameters for the image; a first refinement of the current virtual-camera parameters for the image; and a second refinement of the current virtual-camera parameters for the image.

5. The dental-treatment monitoring system of claim 4 , further comprising, for each optimization iteration, thresholding the image to generate a first teeth mask for the image; thresholding a next corresponding image generated from the time-projected three-dimensional model of the patient's teeth using the current virtual-camera parameters for the image to generate a next second corresponding teeth mask, generating a distance transform of the first teeth mask, searching over scalings, rotations, and translations of the next second corresponding teeth mask to identify a next minimum-cost overlay of the next second corresponding teeth mask over the distance transform of the first teeth mask, and adjusting the virtual-camera parameters for the image corresponding to the next minimum-cost overlay; and selecting, as the adjusted virtual-camera parameters for the image, the virtual-camera parameters associated with lowest-cost overlay of any of the generated next second corresponding teeth mask over the distance transform of the first teeth mask.

6. The dental-treatment monitoring system of claim 5 wherein the cost of an overlay of a next second corresponding teeth mask over the distance transform of the first teeth mask is the sum of the distances associated with pixels in the distance transform of the first teeth underlying a tooth region within the next second corresponding teeth mask.

7. The dental-treatment monitoring system of claim 5 wherein adjusting the virtual-camera parameters for the image corresponding to the next minimum-cost overlay includes: adjusting the virtual-camera parameters to move the position of the center of the next corresponding image generated from the time-projected three-dimensional model of the patient's teeth; adjusting the virtual-camera parameters to rotate the next corresponding image generated from the time-projected three-dimensional model of the patient's teeth in the image plane; adjusting the virtual-camera parameters to rotate the next corresponding image generated from the time-projected three-dimensional model of the patient's teeth about an axis parallel to the image plane; adjusting the virtual-camera parameters to rescale the next corresponding image generated from the time-projected three-dimensional model of the patient's

Assignees

• Align Technology Inc

Inventors

• Borovinskih Artem
• Derakhshan Mitra
• Koppers Carina
• Meyer Eric
• Tolstaya Ekaterina
• Brailov Yury

Classifications

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T7/0016

  involving temporal comparison · CPC title

• A61C7/002 Primary

  Orthodontic computer assisted systems · CPC title

• G06T7/0014

  using an image reference approach · CPC title

• G06T7/32

  using correlation-based methods · CPC title