Inspection workflow using object recognition and other techniques

Patent metadata

Field	Value
Publication number	US-12088910-B2
Application number	US-202117348755-A
Country	US
Kind code	B2
Filing date	Jun 15, 2021
Priority date	Nov 6, 2017
Publication date	Sep 10, 2024
Grant date	Sep 10, 2024

Abstract

Systems for guiding a user through a workflow routine can include an inspection tool, a user interface, memory, and a processor. The processor can provide instructions via the user interface to perform a workflow routine using the inspection tool and save acquiring inspection data to memory. Instructions can direct a user to which equipment to inspect and/or how to collect inspection data associated with one or more pieces of equipment. Systems can determine which equipment is available for inspection by the user, such as via image recognition or proximity detection, and instruct a user to acquire inspection data associated with such equipment. Workflow routine instructions can be provided to the user via various devices, such as an inspection tool, smartphone, or a tablet.

First claim

The invention claimed is:

1. A system, comprising:
a user interface;
a processor communicatively coupled to the user interface;
a location device to detect a location of the system and output location data;
a non-transitory computer readable medium configured to store executable instructions, wherein the executable instructions, when executed by the processor, cause the system to:
receive an inspection workflow for a plurality of pieces of equipment;
detect, based on first location data indicating locations of the plurality of pieces of equipment, and second location data indicating the location of the system by the location device, that pieces of equipment among the plurality of pieces of equipment are within a predetermined proximity of the system;
present, in the user interface, a list of the pieces of equipment that are within the predetermined proximity of the system;
determine that maintenance for a given piece of equipment in the list has been performed and update the user interface to indicate which of the pieces of equipment in the list has been analyzed according to the inspection workflow and which of the pieces of equipment in the list has yet to be analyzed; and
in response to user selection of a piece of equipment identified in the user interface, present, via the user interface, an indication of guidance for inspection of the piece of equipment.

2. The system of claim 1 , wherein the executable instructions, when executed by the processor, further cause the system to:
retrieve first location data of the piece of equipment; and
provide a direction to the piece of equipment based on the first location data and second location data detected by the location device.

3. The system of claim 1 , wherein the executable instructions, when executed by the processor, further cause the system to:
collect measurement data from the piece of equipment; and
associate the measurement data with the piece of equipment.

4. The system of claim 1 , wherein the executable instructions, when executed by the processor, further cause the system to:
collect image data in real time, wherein the image data is acoustic image data, infrared image data, or visible light image data.

5. The system of claim 1 , further comprising an imaging tool, wherein the executable instructions, when executed by the processor, further cause the system to:
retrieve a first reference image of the piece of equipment;
display, in the user interface, the first reference image as a template image;
present, in the user interface, a live image to assist in positioning the imaging tool to capture an image of the piece of equipment corresponding to the template image;
capture the image of the piece of equipment as a second reference image; and
associate the second reference image with the piece of equipment.

6. The system of claim 5 , wherein the viewing perspective comprises an orientation, a position, or a measuring distance.

7. A method, comprising:
receiving an inspection workflow for a plurality of pieces of equipment;
detecting that pieces of equipment among the plurality of pieces of equipment are within a predetermined proximity of a user, wherein the detecting is based on first location data indicating locations of the plurality of pieces of equipment, and second location data indicating a location of the user;
present, in a user interface, a list of the pieces of equipment that are within the predetermined proximity of the user;
indicate, in the user interface, which of the pieces of equipment in the list has been analyzed according to the inspection workflow and which of the pieces of equipment in the list has yet to be analyzed; and
in response to user selection of a piece of equipment identified in the user interface, present, via the user interface, an indication of guidance for inspection of the piece of equipment.

8. The method of claim 7 , further comprising:
collecting measurement data from the piece of equipment; and
associating the measurement data with the piece of equipment.

9. The method of claim 7 , further comprising:
retrieving a first reference image of the piece of equipment;
displaying the first reference image as a template image;
presenting a live image to assist in positioning an imaging tool to capture an image of the piece of equipment corresponding to the template image;
capturing the image of the piece of equipment as a second reference image; and
associating the second reference image with the piece of equipment.

10. The method of claim 9 , further comprising:
updating the user interface to provide updated information indicating which of the pieces of equipment has been maintained according to the inspection workflow and which equipment has yet to be maintained.

11. A non-transient computer-readable medium containing executable instructions, wherein the executable instructions, when executed by a processor of a system, cause the system to:
receive an inspection workflow for a plurality of pieces of equipment;
detect that pieces of equipment among the plurality of pieces of equipment are within a predetermined proximity of a user, wherein the pieces of equipment are detected within the predetermined proximity of the user based on first location data indicating locations of the plurality of pieces of equipment, and second location data indicating a location of the user;
present, in a user interface, a list of the pieces of equipment that are within the predetermined proximity of the user;
indicate, in the user interface, which of the pieces of equipment in the list has been analyzed according to the inspection workflow and which of the pieces of equipment in the list has yet to be analyzed; and
in response to user section of a piece of equipment identified in the user interface, present, via the user interface, guidance related to the inspection workflow for inspection of the piece of equipment.

12. The computer-readable medium of claim 11 , wherein the executable instructions, when executed by the processor, further cause the system to:
collect measurement data from the piece of equipment; and
associate the measurement data with the piece of equipment.

13. The computer-readable medium of claim 11 , wherein the executable instructions, when executed by the processor, further cause the system to:
in response to detecting only one piece of equipment within the predetermined proximity of the user, automatically select the only one piece of equipment for inspection; and
present, via the user interface, the guidance for the only one piece of equipment.

Assignees

• Fluke Corp

Inventors

• Bergstrom Peter A
• Knight Brian
• Rhead Jamie

Classifications

• H04N7/183

  for receiving images from a single remote source · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06Q10/20 Primary

  Administration of product repair or maintenance · CPC title

• H04N23/64 Primary

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title