Methods and systems for enhanced automated visual inspection of a physical asset

Patent metadata

Field	Value
Publication number	US-9251582-B2
Application number	US-201213731209-A
Country	US
Kind code	B2
Filing date	Dec 31, 2012
Priority date	Dec 31, 2012
Publication date	Feb 2, 2016
Grant date	Feb 2, 2016

Abstract

A computer-implemented system for enhanced automated visual inspection of a physical asset includes a visual inspection device capable of generating images of the physical asset and a computing device including a processor and a memory device coupled to the processor. The computing device includes a storage device coupled to the memory device and coupled to the processor. The storage device includes at least one historic image of the physical asset and at least one engineering model substantially representing the physical asset. The computing device is configured to receive, from a present image source, at least one present image of the physical asset captured by the visual inspection device. The computing device is configured to identify at least one matching historic image corresponding to the at least one present image. The computing device is configured to identify at least one matching engineering model corresponding to the at least one present image.

First claim

What is claimed is:

1. A computer-implemented system for enhanced automated visual inspection of a physical asset comprising:
a visual inspection device that generates images of the physical asset; and
a computing device including a processor, a memory device coupled to said processor, and a storage device coupled to said memory device and to said processor, wherein said storage device includes at least one historic image of the physical asset and at least one engineering model substantially representing the physical asset, said computing device configured to:
receive, from a present image source, at least one present image of the physical asset captured by said visual inspection device;
determine, from said at least one present image, a set of present salient features;
determine, from said at least one historic image, a set of historic salient features;
compare said set of present salient features and said set of historic salient features;
identify from said at least one historic image using at least one image retrieval method employing said set of present salient features and said set of historic salient features, at least one matching historic image corresponding to said at least one present image;
identify from said at least one engineering model, at least one matching engineering model corresponding to said at least one present image; and
determine a pose of said visual inspection device with respect to the physical asset based upon said engineering model.

2. A computer-implemented system in accordance with claim 1 , wherein said present image source is at least one of:
said visual inspection device;
said storage device;
said memory device; and
an external storage device.

3. A computer-implemented system in accordance with claim 1 , further configured to use keypoint matching to identify the best sets of matching points between said at least one historic image and said at least one present image.

4. A computer-implemented system in accordance with claim 1 , further configured to:
determine two-dimensional pixel locations in said at least one matching historic image;
determine, from said two-dimensional pixel locations, three-dimensional point locations in said matching engineering model;
generate, from said three-dimensional point locations, a mathematical model of projections of said matching engineering model, said mathematical model representative of a function of characteristics of said visual inspection device, rotation of said visual inspection device, and translation of said visual inspection device; and
determine, from said mathematical model of projections of said matching determine, from said mathematical model of projections of said matching engineering model and said at least one present image, a position of said visual inspection device with respect to the physical asset.

5. A computer-implemented system in accordance with claim 4 , wherein said computer-implemented system configured to identify at least one matching engineering model corresponding to said at least one present image is further configured to:
create, using said position of said visual inspection device, a projection of said matching engineering model;
overlay said projection of said at least one present image on said matching engineering model; and
measure differences between said projection and said at least one present image.

6. A computer-based method for enhanced automated visual inspection of a physical asset, wherein said method is performed by a computing device, said computing device including a processor, a memory device coupled to said processor, and a storage device coupled to said memory device and to said processor, wherein said storage device includes at least one historic image of the physical asset and at least one engineering model substantially representing the physical asset, said method comprising:
receiving, from a present image source, at least one present image of the physical asset captured by said visual inspection device, wherein said present image source is at least one of: said visual inspection device; said storage device; said memory device; and an external storage device;
determining, from said at least one present image, a set of present salient features;
determining, from said at least one historic image, a set of historic salient features;
comparing said set of present salient features and said set of historic salient features;
identifying from said at least one historic image using at least one image retrieval method employing said set of present salient features and said set of historic salient features, at least one matching historic image corresponding to said at least one present image;
identifying from said at least one engineering model, at least one matching engineering model corresponding to said at least one present image; and
determining a pose of said visual inspection device with respect to the physical asset based upon said engineering model.

7. A computer-based method in accordance with claim 6 , further comprising using keypoint matching to identify the best sets of matching points between said at least one historic image and said at least one present image.

8. A computer-based method in accordance with claim 6 , further comprising:
determining two-dimensional pixel locations in said at least one matching historic image;
determining, from said two-dimensional pixel locations, three-dimensional point locations in said matching engineering model;
generating, from said three-dimensional point locations, a mathematical model of projections of said matching engineering model, said mathematical model representative of a function of characteristics of said visual inspection device, rotation of said visual inspection device, and translation of said visual inspection device; and
determining, from said mathematical model of projections of said matching engineering model and said at least one present image, a position of said visual inspection device with respect to the physical asset.

9. A computer-based method in accordance with claim 8 , wherein identifying at least one matching engineering model corresponding to said at least one present image further comprises:
creating, using said position of said visual inspection device, a projection of said engineering model;
overlaying said at least one present image on said engineering model; and
measuring differences between said projection and said at least one present image.

10. A computer for enhanced automated visual inspection of a physical asset, said computer includes a processor, a memory device coupled to said processor, and a storage device coupled to said memory device and to said processor, wherein said storage device includes at least one historic image of the physical asset and at least one engineering model substantially representing the physical asset, said computer configured to:
receive, from a present image source, at least one present image of the physical asset captured by said visual inspection device;
determine, from said at least one present image, a set of present salient features;
determine, from said at least one historic image, a set of historic salient features;
compare said set of present salient features and said set of historic salient features;
identify from said at least one historic image using at least one image retrieval method employing said set of present salient features and said set of historic salient features, at least one matching historic image corresponding to said at least one present image;
identify from said at least one engineering model, at least one matching engineering model corresponding to said at least one present image; and
determine a pose

Assignees

• Gen Electric

Inventors

• Lim Ser Nam
• Kumar Vinod Padmanabhan
• Irving Russell Robert
• Laflen John Brandon
• Guan Li
• Liu Shubao
• Yang Xingwei

Classifications

• G06T7/001 Primary

  using an image reference approach · CPC title

• G06F17/30247

  Physics · mapped topic

• G06T2207/30244

  Camera pose · CPC title

• G06T7/0028

  Physics · mapped topic

• G06T2207/30164

  Workpiece; Machine component · CPC title