Planar contour recognition method and apparatus, computer device, and storage medium

Patent metadata

Field	Value
Publication number	US-12412279-B2
Application number	US-202217956364-A
Country	US
Kind code	B2
Filing date	Sep 29, 2022
Priority date	Sep 1, 2020
Publication date	Sep 9, 2025
Grant date	Sep 9, 2025

Abstract

This application relates to a planar contour recognition method and apparatus, a computer device, and a storage medium. The method includes obtaining a target frame image collected from a target environment; fitting edge points of an object plane in the target frame image and edge points of a corresponding object plane in a previous frame image to obtain a fitting graph, the previous frame image being collected from the target environment before the target frame image; deleting edge points that do not appear on the object plane of the previous frame image, in the fitting graph; and recognizing a contour constructed by remaining edge points in the fitting graph as a planar contour.

First claim

What is claimed is:

1. A planar contour recognition method, performed by a computer device, the method comprising:
obtaining a target frame image collected from a target environment, the target frame image comprising depth information;
fitting edge points of an object plane in the target frame image and edge points of a corresponding object plane in a previous frame image to obtain a fitting graph, the previous frame image being collected from the target environment before the target frame image, the edge points being three-dimensional edge points;
deleting edge points that do not appear on the object plane of the previous frame image in the fitting graph;
recognizing a contour constructed by remaining edge points in the fitting graph as a planar contour; and
controlling, based on the recognized planar contour, movement of a robot or placement of a target object,
wherein the fitting the edge points of the object plane in the target frame image and the edge points of the corresponding object plane in the previous frame image to obtain the fitting graph comprises: determining a spatial position corresponding to each point in the target frame image according to the depth information; determining a plane on which each point is located based on the spatial position and a plane equation to obtain the object plane; and fitting the edge points of the object plane and the edge points of the object plane corresponding to the previous frame image to obtain the fitting graph, and
wherein the method further comprises: mapping the three-dimensional edge points into two-dimensional edge points; determining a convex polygon corresponding to the two-dimensional edge points, and calculating an area of the convex polygon; determining a bounding graph of the two-dimensional edge points, and calculating an area of the bounding graph; and determining that the object plane is a partial region plane of an object or blocked by another object in the target frame image if a ratio of the area of the convex polygon to the area of the bounding graph reaches a ratio.

2. The method according to claim 1 , the method further comprising:
displaying the target frame image;
displaying the edge points of the object plane in the target frame image and the edge points of the corresponding object plane in the previous frame image on the object plane when fitting the edge points; and
displaying the planar contour constructed by the remaining edge points in the fitting graph on the object plane of the target frame image after recognizing the contour.

3. The method according to claim 2 , wherein the fitting the edge points of the object plane and the edge points of the object plane corresponding to the previous frame image to obtain the fitting graph comprises:
determining a previous target frame image that comprises the object plane in each of the previous frame images if the object plane is the partial region plane of the object or blocked by another object in the target frame image;
extracting edge points from the object plane in the previous target frame image;
selecting target edge points from the edge points that are extracted from the object plane in the previous target frame image and the edge points of the object plane in the target frame image; and
fitting the selected target edge points to obtain the fitting graph.

4. The method according to claim 1 , further comprising:
determining a moving path in each planar contour constructed by using the target frame image, or selecting a landing point in the planar contour; and
generating instructions instructing the robot to move according to the moving path or to land on the landing point.

5. The method according to claim 1 , wherein the object plane is a region of a plane corresponding to the target object, and the method further comprises:
determining a size, an orientation, and a spatial position of the target object according to the planar contour;
clamping the target object based on the size, orientation, and spatial position of the target object; and
placing the clamped target object at a designated position.

6. The method according to claim 5 , wherein the placing the clamped target object at a designated position comprises:
collecting, in a process of placing the target object, an environment image at the designated position;
fitting edge points of each target object plane in the environment image and edge points of a corresponding target object plane in a previous frame environment image to obtain a target fitting graph;
deleting edge points that do not appear on the target object plane of the previous frame environment image in the target fitting graph, the previous frame environment image being a frame image collected before the environment image;
constructing a target planar contour by using remaining edge points in the target fitting graph;
determining a placement posture of the target object according to the target planar contour; and
placing the target object on the target object plane according to the placement posture.

7. The method according to claim 1 , wherein the target frame image comprises a graphic code carrying direction information, and the determining the spatial position corresponding to each point in the target frame image according to the depth information comprises:
determining a coordinate system reference direction according to the direction information carried by the graphic code;
constructing a spatial coordinate system based on the coordinate system reference direction; and
determining the spatial position corresponding to each point in the target frame image based on the depth information in the spatial coordinate system.

8. The method according to claim 1 , wherein the fitting the edge points of the object plane and the edge points of the object plane corresponding to the previous frame image to obtain the fitting graph comprises:
determining a previous target frame image that comprises the object plane in each of the previous frame images if the object plane is the partial region plane of the object or blocked by another object in the target frame image;
extracting edge points from the object plane in the previous target frame image;
selecting target edge points from the edge points that are extracted from the object plane in the previous target frame image and the edge points of the object plane in the target frame image; and
fitting the selected target edge points to obtain the fitting graph.

9. The method according to claim 8 , wherein the selecting target edge points from the edge points that are extracted from the object plane in the previous target frame image and the edge points of the object plane in the target frame image comprises:
determining a first weight corresponding to the target frame image and a second weight corresponding to the previous target frame image, the first weight being different from the second weight;
selecting, according to the first weight, a first target edge point from the edge points of the object planes in the target frame image, and selecting, according to the second weight, a second target edge point from the edge points that are extracted from the object plane in the previous target frame image; and
using the first target edge point and the second target edge point as the target edge points.

10. The method according to claim 1 , further comprising:
determining a size of the fitting graph;
re-obtaining the target frame image collected from the target environment if the size is less than a threshold value; and
deleting edge points that do not appear on the object plane of the previous frame image if the size is greater than or equal to the threshold value in the fitting graph.

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Zhang Shenghao
• Ling Yonggen
• Chi Wanchao
• Zheng Yu
• Jiang Xinyang

Classifications

• G06T7/13 Primary

  Edge detection · CPC title

• G06T7/75

  involving models · CPC title

• G06T7/246

  using feature-based methods, e.g. the tracking of corners or segments · CPC title

• G06T2207/10028

  Range image; Depth image; 3D point clouds · CPC title

• G06T2207/30244

  Camera pose · CPC title