Image segmentation for automatic direct measurement and scanning

Patent metadata

Field	Value
Publication number	US-12561935-B2
Application number	US-202318216059-A
Country	US
Kind code	B2
Filing date	Jun 29, 2023
Priority date	Jun 29, 2023
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Measurement devices such as total stations or laser scanners equipped with camera sensors can capture an image of an environment. An image segmentation model can identify and segment environmental features in the image. The measurement device can display an overlay of the segmented features on top of the image. When a user selects a particular feature, the measurement device can perform point measurements such as scanning to measure a portion of the environment associated with the selected feature. Features in the environment not selected by the user may not be included in the point measurements.

First claim

What is claimed is:

1 . A method for operating a measurement device, the method comprising:
detecting, by the measurement device, an image of an environment; inputting, by the measurement device, the image of the environment into an image segmentation model, the image segmentation model configured to identify a plurality of features in the environment present in the image; determining, by the measurement device based on an output from the image segmentation model, a plurality of bounding boxes, each bounding box of the plurality of bounding boxes outlining a feature of the plurality of features in the environment and determined based on a plurality of pixels in the image at a boundary of the feature; presenting, by the measurement device, an identification of the plurality of features for display to a user, including:
presenting, by the measurement device, the image of the environment for display to the user; and
presenting, by the measurement device, an overlay of the plurality of bounding boxes over the display of the image of the environment to the user;
receiving, by the measurement device, a selection of a desired feature of the plurality of features from the user; and performing, by the measurement device, point measurements of a portion of the environment associated with the desired feature based on a bounding box of the plurality of bounding boxes associated with the desired feature, wherein features in the environment not selected by the user are not included in the point measurements.

2 . The method of claim 1 , wherein receiving the selection of the desired feature further comprises:
receiving, by the measurement device, the selection of the bounding box on the overlay from the user.

3 . The method of claim 1 , further comprising:
identifying, by the measurement device, a portion of the bounding box that has been measured in previous point measurements; outputting, to the user, a notification of the previously measured portion in the bounding box; receiving, from the user, another selection of the bounding box; and in response to receiving the other selection, performing, by the measurement device, the point measurements of the portion of the environment associated with the feature based on the bounding box.

4 . The method of claim 1 , wherein performing the point measurements of the portion of the environment associated with the desired feature further comprises:
identifying, by the measurement device, a resolution associated with a type of the desired feature; determining, by the measurement device, the portion of the environment to measure based on the bounding box; and performing, by the measurement device, the point measurements for the portion of the environment using the resolution.

5 . The method of claim 1 , wherein the portion of the environment comprises a region associated with the desired feature outlined by the bounding box and a buffer region surrounding the desired feature, wherein a thickness of the buffer region is selected by the user.

6 . The method of claim 1 , wherein the image segmentation model is a trained machine learning model.

7 . A measurement device for performing point measurements, the measurement device comprising:
one or more memory devices coupled with the measurement device containing instructions that, when executed by one or more processors, perform the following steps:
detecting an image of an environment;
inputting the image of the environment into an image segmentation model, the image segmentation model configured to identify a plurality of features in the environment present in the image;
determining, based on an output from the image segmentation model, a plurality of bounding boxes, each bounding box of the plurality of bounding boxes outlining a feature of the plurality of features in the environment and determined based on a plurality of pixels in the image at a boundary of the feature;
presenting an identification of the plurality of features for display to a user, including:
presenting the image of the environment for display to the user; and
presenting an overlay of the plurality of bounding boxes over the display of the image of the environment to the user;

receiving a selection of a desired feature of the plurality of features from the user; and
performing point measurement of a portion of the environment associated with the desired feature based on a bounding box of the plurality of bounding boxes associated with the desired feature, wherein features in the environment not selected by the user are not included in the point measurements.

8 . The measurement device of claim 7 , wherein the instructions further cause the one or more processors to receive the selection of the feature by:
receiving the selection of the bounding box on the overlay from the user.

9 . The measurement device of claim 7 , wherein the instructions further cause the one or more processors to:
identify a portion of the bounding box that has been measured in previous point measurements; output, to the user, a notification of the previously measured portion in the bounding box; receive, from the user, another selection of the bounding box; and in response to receiving the other selection, perform the point measurements of the portion of the environment associated with the feature based on the bounding box.

10 . The measurement device of claim 7 , wherein the instructions further cause the one or more processors to perform the point measurements of the portion of the environment associated with the desired feature by:
identifying a resolution associated with a type of the desired feature; determining the portion of the environment to measure based on the bounding box; and performing the point measurements for the portion of the environment using the resolution.

11 . The measurement device of claim 7 , wherein the portion of the environment comprises a region associated with the desired feature outlined by the bounding box and a buffer region surrounding the desired feature, wherein a thickness of the buffer region is selected by the user.

12 . The measurement device of claim 7 , wherein the image segmentation model is a trained machine learning model.

13 . A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors, perform the following steps:
detecting an image of an environment; inputting the image of the environment into an image segmentation model, the image segmentation model configured to identify a plurality of features in the environment present in the image; determining, based on an output from the image segmentation model, a plurality of bounding boxes, each bounding box of the plurality of bounding boxes outlining a feature of the plurality of features and determined based on a plurality of pixels in the image at a boundary of the feature; presenting an identification of the plurality of features for display to a user, including:
presenting the image of the environment for display to the user; and
presenting an overlay of the plurality of bounding boxes over the display of the image of the environment to the user;
receiving a selection of a desired feature of the plurality of features from the user; and performing point measurements of a portion of the environment associated with the desired feature based on a bounding box of the plurality of bounding boxes associated with the desired feature, wherein features in the environment not selected by the user are not included in the point measurements.

14 . The non-transitory computer-readable medium of claim 1

Assignees

• Trimble Inc

Inventors

• Clin Guillaume
• Ralston Stuart Edward
• White Dylan

Classifications

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06F2203/04804

  Transparency, e.g. transparent or translucent windows · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06T2207/20104

  Interactive definition of region of interest [ROI] · CPC title