Quality assured manufacturing

Patent metadata

Field	Value
Publication number	US-2015253766-A1
Application number	US-201514639889-A
Country	US
Kind code	A1
Filing date	Mar 5, 2015
Priority date	Mar 6, 2014
Publication date	Sep 10, 2015
Grant date	—

Abstract

The invention relates to a tool location system for a handheld tool in a manufacturing environment, wherein the tool is equipped with a camera for capturing images. According to the invention, the location system comprises numerical evaluator means, which is built to determine a tool-location of the tool by a simultaneous location and mapping (SLAM) navigation calculation based on the images from the camera. Therein the tool-location is determined with respect to a global coordinate system in relation to the manufacturing environment from portions of the images which comprise a view of the manufacturing environment and the tool-location is determined with respect to a local coordinate system in relation to the workpiece from portions of the images which comprise a view of the workpiece.

First claim

What is claimed is:

1 . A tool location system for a handheld tool in a manufacturing environment where a workpiece is manufactured, the tool comprising:
a camera for capturing images; and a numerical evaluator means configured to determine a tool-location of the tool by a simultaneous location and mapping (SLAM) navigation calculation based on the images from the camera; wherein the tool-location is determined with respect to a global coordinate system in relation to the manufacturing environment from portions of the images which comprise a view of the manufacturing environment, and wherein the tool-location is determined with respect to a local coordinate system in relation to the workpiece from portions of the images which comprise a view of the workpiece,

2 . The system according to claim 1 , wherein the multiple manufacturing stations is located within a car body, an airplane, or a machinery production.

3 . The system according to claim 1 , wherein the local coordinate system is moving with respect to the global coordinate system in an assembly line arrangement.

4 . The system according to claim 1 , wherein:
the determination of the tool-location involves a matching of the SLAM navigation with known spatial information of the workpiece and/or the environment, of spatial information in form of CAD or pointcloud data.

5 . The system according to claim 1 , wherein:
the evaluator means is configured to establish a seamless transition of the tool-location between the local coordinate system and the global coordinate system, with a live-updated referencing of the local coordinate system with respect to the global coordinate system.

6 . The system according to claim 1 , wherein:
the tool comprise an inertial measurement unit (IMU) and the determination of the tool-location involves information gathered from the IMU used in an inertial navigation algorithm.

7 . The system according to claim 1 , wherein:
the manufacturing environment and/or the workpiece comprises multiple cooperative and or actively light emitting targets built to be identified in the camera image, contrast faces, reflectors or LEDs.

8 . The system according to claim 1 , wherein:
the tool comprises a communication interface, which is built to exchange its localization and/or image information with a manufacturing management system and/or other tools.

9 . The system according to claim 8 , wherein
the tool comprises a sensor means for sensing a work process parameter of a tools operation, and the work process parameter is provided together with the corresponding tool-location as tool-surveillance data by the communication interface, wherein the tool-surveillance data comprises an image from the camera depicting a work process result of the tools operation.

10 . The system according to claim 1 , wherein:
the tool is a mounting tool selected from the list consisting of a screwdriver tool, a wrench tool, a clamping tool, and a rivet-tool, driven by electricity or compressed air, and the camera has a field of view comprising at least a part of an operating range of the tool.

11 . The system according to claim 1 , wherein:
the workpiece is movable and is also equipped with a camera for SLAM navigation of the workpiece-location in the global coordinate system, wherein the workpiece-location and the tool-location are combined, wherein a referencing of the global coordinate system and the local coordinate system is established, wherein a back-referencing between the tool and the workpiece navigation is established by the location of the tool with the camera at the workpiece and the location of the workpiece with the camera at the tool.

12 . A tool surveillance system for a manufacturing environment, wherein the manufacturing environment comprises stationary items and movable items, wherein the tool surveillance system comprises:
a workpiece to be manufactured and/or parts thereof, a tool to be handled by a human worker, and a tool location system according to claim 1 for the tool.

13 . The tool surveillance system claim 12 , wherein the manufacturing environment is equipped with:
multiple global cameras for imaging the manufacturing environment, and an image processing means built to identify/detect the stationary and movable items in the manufacturing environment and to determine their global positional coordinates in a global environment coordinate system, in at least three dimensions, and wherein the global positional coordinates are matched with the determined tool-location, wherein a back-referencing between the tool and the environment navigation is established by the location of the tool with the camera at the environment and the location of the environment with the camera at the tool.

14 . A method for locating a handheld tool in a manufacturing environment where a workpiece is manufactured, wherein the tool is equipped with a camera for capturing images, the method comprising:
determining a tool-location of the tool by a SLAM navigation calculation based on the images from the camera, that further comprises:
determining the tool-location with respect to a global coordinate system in relation to the manufacturing environment from portions of the image which comprise a view of the manufacturing environment, and
determining the tool-location with respect to a local coordinate system in relation to the workpiece from portions of the image which comprise a view of the workpiece;
wherein the local coordinate system is moving with respect to the global coordinate system.

15 . The method according to claim 14 , wherein:
in an environment with multiple manufacturing stations of a car body, airplane or machinery production, in an assembly line arrangement,

16 . The method according to claim 14 , wherein:
determining the tool-location involves a matching of the SLAM navigation with known spatial information of the workpiece and/or the environment, of spatial information in form of CAD or pointcloud data.

17 . The method according to claim 12 , wherein
a sensing of a work process parameter by a sensor means at the tool and providing the tool-location together with the corresponding work process parameter as tool-surveillance data by a communication interface, with providing the surveillance data to a manufacturing management system for managing the manufacturing and/or documenting the manufacturing of the workpiece, together with an image from the camera of a work process result at the corresponding tool-location.

18 . One or more non-transitory computer-readable media storing one or more programs that are configured, when executed, to cause one or more processors to execute a SLAM navigation algorithm based on images from a camera at a tool in a manufacturing environment in which a workpiece is manufactured, with a moving workpiece in an assembly line configuration, wherein a location of the tool is established:
with respect to a global coordinate system relative to the environment by portions of the images depicting the environment and also with respect to a local coordinate system relative to the workpiece by portions of the image depicting the workpiece.

Assignees

• Hexagon Technology Ct Gmbh

Inventors

• Pettersson Bo
• Siercks Knut
• Zebhauser Benedikt

Classifications

• G05B19/4187 Primary

  by tool management · CPC title

• G05B2219/45127

  Portable, hand drill · CPC title

• G05B2219/31304

  Identification of workpiece and data for control, inspection, safety, calibration · CPC title

• G05B2219/40557

  Tracking a tool, compute 3-D position relative to camera · CPC title

• G05B2219/31027

  Computer assisted manual assembly CAA, display operation, tool, result · CPC title