Object tracking-based control of manufacturing processes in the metalworking industry

Patent metadata

Field	Value
Publication number	US-11460832-B2
Application number	US-202016815456-A
Country	US
Kind code	B2
Filing date	Mar 11, 2020
Priority date	Sep 12, 2017
Publication date	Oct 4, 2022
Grant date	Oct 4, 2022

Abstract

A method for manufacturing control of machining of workpieces in a manufacturing hall that includes providing a plurality of image acquisition devices configured to acquire image sequences of allocated observation areas, identifying an object to be tracked in a first image of a first image sequence using at least one object feature, determining positions of the object feature in images of subsequent image sequences, determining a current position of the object by localizing the object feature in a temporally current image of one of the image sequences, and assigning the position of the object feature in the temporally current image to a current position in the observation area of the image sequence that includes the temporally current image, and integrating the determined current position of the object into the manufacturing control of the hall.

First claim

What is claimed is:

1. A method of manufacturing control of processing of workpieces according to processing plans in a manufacturing hall for the manufacture of an end product, the method comprising:
providing in the manufacturing hall a plurality of image acquisition devices configured to acquire image sequences of allocated observation areas within the manufacturing hall;
identifying an object to be tracked in a first image of the image sequences using at least one object feature in the first image;
determining a processing plan that relates to the object to be tracked;
determining positions of the object feature in image sequences acquired temporally after the first image;
determining a current position of the object to be tracked by localizing the object feature in a temporally current image of one of the image sequences and assigning the current position of the object feature in the temporally current image to a current position within the observation area of the image sequence that includes the temporally current image; and
integrating the determined current position of the object to be tracked into a manufacturing control of the manufacturing hall for manufacturing the end product.

2. The method of claim 1 , wherein the determined current position of the object to be tracked is integrated into the manufacturing control by using the determined processing plan.

3. The method of claim 1 , wherein the object to be tracked is tracked within the images on the basis of pixel changes associated with an object feature.

4. The method of claim 1 , wherein the object to be tracked in the first image is identified with a software system.

5. The method of claim 1 , wherein localizing the object feature comprises:
determining an object feature vector describing a movement of the object feature between images of the first image sequence; and
projecting the object feature vector into an image of a second of the image sequences.

6. The method of claim 5 , further comprising, when the object leaves an observation area within the manufacturing hall, checking whether a new object feature that corresponds to the object feature and the object feature vector, can be identified in a further image sequence.

7. The method of claim 5 , wherein the object feature vector is a direction vector or a velocity vector.

8. The method of claim 5 , wherein the observation areas of the first image sequence and the second image sequence have an overlapping area and a hand-over of the object features from the first image sequence to the second image sequence takes place on the basis of pixel values in the overlapping area of images at approximately the same point in time.

9. The method of claim 1 , wherein the object to be tracked is a workpiece, a transport device, a tool, a mobile unit, or a person.

10. The method of claim 1 , wherein the image sequences of the image acquisition devices have a unique time reference for determining position with respect to each other.

11. The method of claim 1 , wherein the object feature includes a specific geometric shape, a color, a light signal, or an extension of an area in the image.

12. The method of claim 1 , wherein assigning of the position of the object feature in the temporally current image to a current position within the observation area of the image sequence comprises:
providing reference points in the observation areas, and
calculating an exact location and point in time of the object to be tracked using at least one of the reference points.

13. The method of claim 1 , further comprising:
providing the determined current position of the object to be tracked via a suitable interface, and
integrating the determined current position of the object to be tracked in a motion analysis of the object to be tracked.

14. The method of claim 1 , further comprising:
defining one or both of zones and spatial gates in the manufacturing hall, and
comparing the determined current position of the object to be tracked with respect to the zones, spatial gates, or both the zones and the spatial gates and controlling the manufacturing based on the comparison.

15. The method of claim 14 , wherein comparing the determined current position of the object to be tracked results in the object to be tracked being located in a zone or having left the zone or having passed a spatial gate.

16. The method of claim 1 , wherein integrating the determined current position of the object to be tracked comprises one or more of:
outputting signals to support localizing the object to be tracked;
displaying information on one or both of the determined current position and the manufacturing status of the object to be tracked;
setting operating parameters of a machine tool;
updating a log that logs the processing steps; or
assigning the object to be tracked to an order for the manufacturing of one or more end products in several different processing steps.

17. The method of claim 1 , wherein the object to be tracked is a workpiece and integrating the determined current position of the object to be tracked comprises controlling or monitoring or both controlling and monitoring:
processing of the workpiece in several different processing steps that are carried out at different locations; or
transporting of the workpiece between different processing steps that are carried out at different locations.

18. The method of claim 1 , wherein the manufacturing control system is supported by an indoor localization method comprising:
providing an indoor location system having a plurality of transceiver units installed in the manufacturing hall, at least one mobile unit, and a localization analysis unit, the transceiver units and the mobile unit being configured for transmitting and receiving electromagnetic signals and the localization analysis unit being configured to determine the positions of the mobile unit in the manufacturing hall from runtimes of the electromagnetic signals between the transceiver units and the mobile unit;
assigning the mobile unit to the object to be tracked;
determining the position of the assigned object by localizing the assigned mobile unit with the indoor location system; and
integrating the determined position into one or both of
(i) an allocation process, in which a pixel structure is determined in the image for the object to be tracked, or
(ii) the manufacturing control for manufacturing the end product.

19. A manufacturing control system for controlling manufacturing processes in a manufacturing hall, comprising:
a plurality of image acquisition devices installed in the manufacturing hall and configured to acquire image sequences of allocated observation areas within the manufacturing hall; and
an analysis unit configured to determine, exchange, and provide data on the position of an object to be tracked in the manufacturing hall and to perform a method comprising:
identifying an object to be tracked in a first image of the image sequences using at least one object feature in the first image;
determining a processing plan that relates to the object to be tracked;
determining positions of the object feature in image sequences acquired temporally after the first image;
determining a current position of the object to be tracked by localizing the object feature in a temporally current image of one of the image sequences and assigning the current position of the object feature in the temporally current image to a current position within the observation area of the image sequence that includes the tempor

Assignees

• Trumpf Werkzeugmaschinen Gmbh Co Kg

Inventors

• Bauer Klaus
• Bauer Christian
• Beuttler Manuel
• Bock Hans-Peter
• Bossert Daniel
• Gomes Marcel Ely
• Fischer Stephan
• Goerg Christian
• Kiefer Manuel
• Larsen-Vefring Thorsten
• Perdana Frank
• Siddiqui Zaigham Faraz
• Wahl Eberhard

Classifications

• G05B2219/31432

  Keep track of conveyed workpiece, batch, tool, conditions of stations, cells · CPC title

• Y02P90/30

  Computing systems specially adapted for manufacturing · CPC title

• G05B2219/32055

  Identify workpiece, read status centrally, machine, adapt status centrally · CPC title

• G06V2201/06

  Recognition of objects for industrial automation · CPC title

• G06Q50/04

  Manufacturing · CPC title