3D segmentation for robotic applications

Patent metadata

Field	Value
Publication number	US-10682764-B2
Application number	US-201715663783-A
Country	US
Kind code	B2
Filing date	Jul 30, 2017
Priority date	Jan 29, 2015
Publication date	Jun 16, 2020
Grant date	Jun 16, 2020

Abstract

A robotic system includes a robot having an associated workspace; a vision sensor constructed to obtain a 3D image of a robot scene including a workpiece located in the workspace; and a control system communicatively coupled to the vision sensor and to the robot. The control system is configured to execute program instructions to filter the image by segmenting the image into a first image portion containing substantially only a region of interest within the robot scene, and a second image portion containing the balance of the robot scene outside the region of interest; and by storing image data associated with the first image portion. The control system is operative to control movement of the robot to perform work, on the workpiece based on the image data associated with the first image portion.

First claim

What is claimed is:

1. A robotic system, comprising:
a robot having an associated workspace;
a vision sensor constructed to obtain a 3D image of a robot scene including a workpiece located in the workspace; and
a control system communicatively coupled to the vision sensor and to the robot, wherein the control system is configured to execute program instructions to filter the image by segmenting the image into a first image portion containing only a region of interest within the robot scene, and a second image portion containing a remainder of the robot scene which is outside the region of interest, and wherein the control system is further configured to store image data associated with the first image portion,
wherein the control system is operative to control movement of the robot to perform work on the workpiece based on the image data associated with the first image portion;
wherein the vision sensor is constructed to provide robot scene image data from the image of the robot scene, and the control system is configured to execute program instructions to transform the robot scene image data to a robot coordinate system; and
wherein the control system is configured to execute the program instructions to filter the image by segmenting the image after the control system transforms the robot scene image data to the robot coordinate system.

2. The robotic system of claim 1 , wherein the control system is configured to execute program instructions to calibrate the vision sensor to a robot coordinate system.

3. The robotic system of claim 1 , wherein the region of interest is the workpiece; and wherein the first image portion includes only the workpiece and a workpiece displacement range.

4. The robotic system of claim 1 , wherein the region of interest is the workpiece; and wherein the first image portion includes only the workpiece.

5. The robotic system of claim 1 , wherein the control system is configured to execute program instructions to create a space band around the workpiece; wherein the region of interest is the workpiece and the space band; and wherein the first image portion includes only the workspace and the space band.

6. The robotic system of claim 5 , wherein the region of interest is the workspace; and wherein the first image portion includes substantially only the workspace.

7. The robotic system of claim 1 , wherein the control system is configured to execute program instructions to create a space band around the workspace; wherein the region of interest is the work piece, the workspace and the space band; and wherein the first image portion includes only the work piece, the workspace and the space band.

8. The robotic system of claim 1 , wherein the vision sensor is mounted on a moving portion of the robot.

9. The robotic system of claim 1 , wherein the region of interest is confined to areas accessible by the robot.

10. A method for operating a robot, comprising:
acquiring a 3D image of a robot scene using a robotic vision sensor;
performing 3D segmenting the image into a first image portion containing only a region of interest within the robot scene, and a second image portion containing a remainder of the robot scene which is outside the region of interest;
storing image data associated with the first image portion;
operating a robot to perform work on a workpiece using the stored image data associated with the first image portion;
wherein the 3D image includes robot scene image data, further comprising transforming the robot scene image data from a robotic vision sensor coordinate system to a robot coordinate system; and
wherein the performing 3D segmenting is performed after the transforming.

11. The method of claim 10 , further comprising defining a space band disposed about the workpiece, wherein the region of interest includes only the workpiece and the space band.

12. The method of claim 11 , wherein the space band is defined as a workpiece displacement range.

13. The method of claim 10 , further comprising defining a space band disposed about the workspace, wherein the region of interest includes the work piece, the workspace and the space band.

14. The method of claim 13 , wherein the space band is defined by a robot tool envelope.

15. The method of claim 10 , wherein the region of interest is confined to areas accessible by the robot.

16. A method for operating a robot, comprising:
calibrating a robotic vision sensor to a robot coordinate system;
acquiring a 3D image of a robot scene using the robotic vision sensor;
filtering the 3D image to yield an image portion containing only a region of interest within the robot scene;
operating a robot to perform work on a workpiece in the region of interest using the first image portion; and
wherein the region of interest is further defined by an area of reach of the robot.

17. The method of claim 16 , further comprising segmenting the image into a first image portion containing only the region of interest within the robot scene, and a second image portion containing a remainder of the robot scene which is outside the region of interest; and storing image data associated with the first image portion.

18. The method of claim 16 , wherein the 3D image includes robot scene image data, further comprising transforming the robot scene image data from a robotic vision sensor coordinate system to a robot coordinate system.

19. The method of claim 16 , wherein the area of reach includes only areas accessible by a tool connected to the robot.

Assignees

• Abb Schweiz Ag

Inventors

• Boca Remus
• Fuhlbrigge Thomas A

Classifications

• G06T2207/20121

  Active appearance model [AAM] · CPC title

• G05B2219/37555

  Camera detects orientation, position workpiece, points of workpiece · CPC title

• G06T2207/20104

  Interactive definition of region of interest [ROI] · CPC title

• G05B2219/40004

  Window function, only a specific region is analyzed · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title