Method and computing systems for performing object detection

Patent metadata

Field	Value
Publication number	US-12558790-B2
Application number	US-202217733079-A
Country	US
Kind code	B2
Filing date	Apr 29, 2022
Priority date	May 4, 2021
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

A computing system including a communication interface and a processing circuit. The communication interface communicates with a robot and a camera having a field of view. The processing circuit performs obtaining image information based on objects within the field of view and determines a first template matching score which indicates a degree of match between the image information and an model template. The processing circuit further determines image edge information based on the image information and determines a second template matching score which indicates a degree of match between the image edge information and a template. The processing circuit additional determines an overall template matching score based on the first template matching score and the second template matching score.

First claim

The invention claimed is:

1 . A computing system comprising:
a communication interface configured to communicate with a robot and with a camera having a camera field of view; at least one processing circuit configured, when one or more objects are or have been in the camera field of view, to perform the following: obtaining image information representing a first object of the one or more objects, wherein the image information is generated by the camera and describes at least an object appearance associated with the first object; determining a first template matching score as an area-based template matching score which indicates a degree of match between a region of the image information and visual description information of a model template, wherein the visual description information describes an object appearance; determining image edge information based on the image information representing the first object, wherein the image edge information identifies a first set of one or more edges corresponding to visual features on the surface of an object that is detected from the image information, or indicates that no edge is detected from the image information; determining template edge information, wherein the template edge information identifies a second set of one or more edges detected from the visual description information of the model template, or indicates that no edge is detected from the visual description information; determining a second template matching score as an edge-based template matching score which indicates a degree of match between the image edge information within the region and the template edge information; determining an overall template matching score as a weighted fraction between the first template matching score and the second template matching score, wherein a first weight for the first template matching score is based in part on one or more reliability factors of the image information including damage to the object represented in the image information; and performing a robot interaction planning operation based on the overall template matching score, wherein the robot interaction planning operation is for planning robot interaction between the robot and the first object.

2 . The computing system of claim 1 , wherein the at least one processing circuit is configured, when the first set of one or more edges are detected from the image information, to generate the image edge information as a first edge bitmap which identifies where the first set of one or more edges are located, and
wherein the at least one processing circuit is configured to generate the second template matching score based on a comparison between the first edge bitmap and a second edge bitmap, wherein the second edge bitmap identifies where a second set of one or more edges are located in the model template based on visual description information.

3 . The computing system of claim 2 , wherein the at least one processing circuit is configured to determine the second template matching score based on a maximum amount by which the first set of one or more edges detected from the image information is overlapping with the second set of one or more edges from the visual description information of the model template.

4 . The computing system of claim 3 , wherein the at least one processing circuit is configured to determine the second template matching score in a manner which is independent of the first template matching score.

5 . The computing system of claim 2 , wherein the first edge bitmap is a first binary bitmap which includes at least area pixels that are assigned a first pixel value,
wherein, when the first set of one or more edges are detected from the image information, the first binary bitmap further includes edge pixels, wherein the edge pixels of the first binary bitmap are pixels thereof that correspond to locations in the image information at which the first set of one or more edges are detected, wherein the edge pixels of the first binary bitmap are assigned a second pixel value, wherein the second edge bitmap is a second binary bitmap which includes at least area pixels that are assigned the first pixel value, and wherein, when the second set of one or more edges are detected from the visual description information of the model template, the second binary bitmap further includes edge pixels, wherein the edge pixels of the second binary bitmap are pixels thereof that correspond to locations in the visual description information at which the second set of one or more edges are detected, and wherein the edge pixels of the second binary bitmap are assigned the second pixel value.

6 . The computing system of claim 5 , wherein the at least one processing circuit is configured to determine the second template matching score based on a maximum amount the edge pixels of the first binary bitmap is overlapping with the edge pixels of the second binary bitmap.

7 . The computing system of claim 5 , wherein the at least one processing circuit is configured to determine, for each edge of the first set of one or more edges, a respective priority level for the edge based on a technique by which the edge is detected from the image information,
wherein the second template matching score is based on one or more respective priority levels associated with the first set of one or more edges.

8 . The computing system of claim 7 , wherein the at least one processing circuit is configured to determine a respective priority level for each edge of the first set of one or more edges based on which one of the following techniques is used to detect the edge: (i) satisfying a defined pixel intensity discontinuity condition or a defined spiked pixel intensity condition at the edge, (ii) satisfying a defined depth discontinuity condition at the edge, or (iii) satisfying a defined orientation discontinuity condition at the edge.

9 . The computing system of claim 2 , wherein the at least one processing circuit is configured to determine the first template matching score based on a degree of match between regions of the image information and corresponding regions of the visual description information.

10 . The computing system of claim 2 , wherein when the visual description information of the model template includes pixels forming a template 2D region, the at least one processing circuit is configured to determine the first template matching score based on a degree of match between the template 2D region and a corresponding image 2D region that is from the image information or is generated based on the image information.

11 . The computing system of claim 10 , wherein the at least one processing circuit is configured to perform an image normalization operation by generating, based on the image information, transformed image information which matches at least one of an object pose, viewpoint, or a lighting condition associated with the visual description information of the model template, and
wherein the corresponding image 2D region is from the transformed image information.

12 . The computing system of claim 10 , wherein the at least one processing circuit is configured to determine the first template matching score based on a size of overlap which indicates how many pixels in the template 2D region satisfy a defined pixel intensity similarity condition when compared to corresponding pixels of the image 2D region.

13 . The computing system of claim 1 , wherein the at least one processing circuit is configured to determine the overall template matching score based on a weighted combination of the first template matching score and the second template matching s

Assignees

• Mujin Inc

Inventors

• Sahloul Hamdi
• Moreira Rodrigues Jose Jeronimo
• Yu Jinze

Classifications

• G06V2201/06

  Recognition of objects for industrial automation · CPC title

• G06V10/752

  Contour matching · CPC title

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/751

  Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching · CPC title

• G06T2207/30241

  Trajectory · CPC title