Automated gauge reading and related systems, methods, and devices

Patent metadata

Field	Value
Publication number	US-11544916-B2
Application number	US-202017097452-A
Country	US
Kind code	B2
Filing date	Nov 13, 2020
Priority date	Nov 13, 2019
Publication date	Jan 3, 2023
Grant date	Jan 3, 2023

Abstract

Computing devices and methods for reading gauges are disclosed. A gauge reading method includes capturing image data corresponding to a captured image of one or more gauges, detecting one or more gauges in the captured image, cropping a detected gauge in the captured image to provide a use image including the detected gauge, and classifying the detected gauge to correlate the detected gauge with a template image. The gauge reading method also includes attempting to perform feature detection rectification on the use image to produce a rectified image of the detected gauge, performing template matching rectification on the use image to produce the rectified image responsive to a failure to perform the feature detection rectification, and estimating a gauge reading responsive to the rectified image. A computing device may implement at least a portion of a gauge reading method.

First claim

What is claimed is:

1. A gauge reading system, comprising:
one or more processors; and
one or more data storage devices including one or more template images of one or more template gauges stored thereon, the one or more data storage devices further including computer-readable instructions stored thereon, the computer-readable instructions configured to instruct the one or more processors to:
classify a detected gauge in a use image to correlate the detected gauge with one of the one or more template images;
attempt to perform feature detection on the use image based on the correlated one of the one or more template images to transform the use image into a rectified image;
perform template matching rectification on the use image to produce the rectified image responsive to a failure to perform the feature detection; and
estimate a gauge reading of the detected gauge responsive to the rectified image.

2. The gauge reading system of claim 1 , wherein the computer-readable instructions are further configured to instruct the one or more processors to detect the detected gauge in a captured image.

3. The gauge reading system of claim 2 , wherein the computer-readable instructions are further configured to instruct the one or more processors to detect a plurality of gauges in the captured image and provide a plurality of sub-images, each of the plurality of sub-images including one of the detected plurality of gauges.

4. The gauge reading system of claim 3 , wherein the computer-readable instructions are configured to instruct the one or more processors to detect the plurality of gauges by detecting substantially rounded regions of lightness surrounded by darkness in the captured image.

5. The gauge reading system of claim 3 , wherein the computer-readable instructions are configured to instruct the one or more processors to detect the plurality of gauges by applying one or more binary thresholds to pixels of the captured image to generate a binary image.

6. The gauge reading system of claim 1 , wherein the computer-readable instructions are configured to instruct the one or more processors to perform the template matching rectification by blurring the use image and generating binary threshold images based on the blurred use image.

7. The gauge reading system of claim 6 , wherein the computer-readable instructions are further configured to instruct the one or more processors to detect contours in the binary threshold images, fit ellipses to the detected contours, and determine a homology estimation to rectify the use image based on the fit of the ellipses to the detected contours.

8. The gauge reading system of claim 1 , wherein the computer-readable instructions are further configured to instruct the one or more processors to estimate the gauge reading by:
estimating a gauge angle of the detected gauge in the rectified image relative to a template angle of a template gauge in the correlated one of the one or more template images; and
estimating a dial angle of a dial of the detected gauge in the rectified image relative to the detected gauge in the rectified image.

9. The gauge reading system of claim 8 , wherein the computer-readable instructions are configured to instruct the one or more processors to estimate the gauge angle by rotating the rectified image by a plurality of different angles and examine a match of the rectified image for each of the plurality of different angles to the correlated one of the one or more template images.

10. The gauge reading system of claim 8 , wherein the computer-readable instructions are configured to instruct the one or more processors to estimate the dial angle by rotating a template image of the dial by a plurality of different angles and examine a match of the rectified image to the template image of the dial at the plurality of different angles.

11. The gauge reading system of claim 1 , further comprising an image sensor configured to provide image data corresponding to a captured image, the use image taken from the captured image.

12. The gauge reading system of claim 11 , further comprising a drone configured to carry the image sensor.

13. The gauge reading system of claim 11 , wherein the image sensor includes one or more of a panoramic camera, a surveillance camera, a still camera, and a video camera.

14. A gauge reading method, comprising:
capturing image data corresponding to a captured image of one or more gauges;
detecting one or more gauges in the captured image;
cropping a detected gauge in the captured image to provide a use image including the detected gauge;
classifying the detected gauge to correlate the detected gauge with a template image;
attempting to perform feature detection rectification on the use image to produce a rectified image of the detected gauge;
performing template matching rectification on the use image to produce the rectified image responsive to a failure to perform the feature detection rectification; and
estimating a gauge reading responsive to the rectified image.

15. The gauge reading method of claim 14 , wherein:
capturing the image data comprises capturing the image data from an angle that is oblique to faces of the one or more gauges; and
performing rectification on the use image comprises resolving a perceived substantially oval shape of a face of the detected gauge in the use image to a substantially circular face in the rectified image.

16. The gauge reading method of claim 14 , wherein detecting the one or more gauges in the captured image comprises:
applying one or more thresholds to the captured image;
blurring the thresholded captured image to generate a plurality of blurred images;
detecting rounded shapes in each of the blurred images;
adding each of the blurred images together to produce a heatmap; and
applying a threshold to the heatmap to produce a binary image including those of the detected rounded shapes detected in a threshold number or greater of the blurred images.

17. The gauge reading method of claim 16 , wherein cropping the detected gauge in the captured image to provide the use image includes;
bounding the detected rounded shapes in the binary image into rectangular regions; and
providing sub-images of the captured image corresponding to the rectangular regions of the binary image, the use image including one of the sub-images.

18. The gauge reading method of claim 14 , wherein attempting to perform the feature detection rectification on the use image includes determining that the feature detection rectification failed responsive to a determination that fewer than a predetermined number of features of the use image match features of the template image.

19. The gauge reading method of claim 14 , wherein performing the template matching rectification on the use image includes:
blurring at least one of a saturation and a value of the use image;
thresholding the blurred use image according to one or more thresholds to provide one or more binary threshold images;
detecting contours in the one or more binary threshold images;
fitting ellipses to the detected contours in the one or more binary threshold images; and
rectifying the use image based on the fit of the ellipses to the detected contours in the one or more binary threshold images to provide the rectified image.

20. The gauge reading method of claim 14 , wherein estimating the gauge reading responsive to the rectified image comprises:
estimating a gauge angle of the detected gauge relative to a template gauge of the template im

Assignees

• Battelle Energy Alliance Llc
• Univ Of Idaho

Inventors

• Al Rashdan Ahmad Y
• Lew Roger
• Wheeler Michael L
• Lancaster Gregory D
• Unruh Troy C

Classifications

• G06V10/247

  by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06V10/28

  Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns · CPC title

• G06V10/443

  by matching or filtering · CPC title

• G06V2201/02

  Recognising information on displays, dials, clocks · CPC title