Method and apparatus for determining position and orientation of bucket of excavator

Patent metadata

Field	Value
Publication number	US-11004235-B2
Application number	US-201916675825-A
Country	US
Kind code	B2
Filing date	Nov 6, 2019
Priority date	Feb 28, 2019
Publication date	May 11, 2021
Grant date	May 11, 2021

Abstract

Embodiments of the present disclosure provide a method and apparatus for determining position and orientation of a bucket of an excavator, an electronic device and a computer readable medium. The method may include: acquiring an image of a bucket of an excavator collected by a camera provided on an excavator body, the image of the bucket including a preset marker provided on the bucket; determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.

First claim

What is claimed is:

1. A method for determining position and orientation of a bucket of an excavator, comprising:
acquiring an image of the bucket of the excavator captured by a camera provided on an excavator body, the image of the bucket comprising a preset marker provided on the bucket;
determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and
converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.

2. The method according to claim 1 , wherein the three-dimensional feature information of the preset marker comprises spatial position information of a feature point of the preset marker; and
the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker comprises:
determining a three-dimensional coordinate of the feature point of the preset marker according to the spatial position information of the feature point of the preset marker;
detecting the feature point of the preset marker in the image of the bucket, and determining a pixel coordinate of the feature point of the preset marker in the image of the bucket; and
estimating the position and orientation information of the camera relative to the bucket on the basis of the three-dimensional coordinate of the feature point of the preset marker, the pixel coordinate of the feature point of the preset marker in the image of the bucket, and a pre-acquired internal parameter of the camera.

3. The method according to claim 2 , wherein the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker further comprises:
matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker.

4. The method according to claim 3 , wherein the matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker comprises:
acquiring angle sensing data sensed by angle sensors provided on an excavating arm of the excavator and the bucket;
determining a target position range of the preset marker in the image of the bucket on the basis of the angle sensing data; and
detecting marker information within the target position range, and matching in the preset marker information library to identify the preset marker.

5. The method according to claim 2 , wherein the preset marker comprises a preset QR code label, and the feature point of the preset marker comprises an angular point of the preset QR code label.

6. An apparatus for determining position and orientation of a bucket of an excavator, comprising:
at least one processor; and
a memory storing instructions, wherein the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising:
acquiring an image of the bucket of the excavator captured by a camera provided on an excavator body, the image of the bucket comprising a preset marker provided on the bucket;
determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and
converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.

7. The apparatus according to claim 6 , wherein the three-dimensional feature information of the preset marker comprises spatial position information of a feature point of the preset marker; and
the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker comprises:
determining a three-dimensional coordinate of the feature point of the preset marker according to the spatial position information of the feature point of the preset marker;
detecting the feature point of the preset marker in the image of the bucket, and determining a pixel coordinate of the feature point of the preset marker in the image of the bucket; and
estimating the position and orientation information of the camera relative to the bucket on the basis of the three-dimensional coordinate of the feature point of the preset marker, the pixel coordinate of the feature point of the preset marker in the image of the bucket, and a pre-acquired internal parameter of the camera.

8. The apparatus according to claim 7 , wherein the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker further comprises:
matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker.

9. The apparatus according to claim 8 , wherein the matching the image of the bucket on the basis of marker information included in a preset marker information library to identify the preset marker comprises:
acquiring angle sensing data sensed by angle sensors provided on an excavating arm of the excavator and the bucket;
determining a target position range of the preset marker in the image of the bucket on the basis of the angle sensing data; and
detecting marker information within the target position range, and matching in the preset marker information library to identify the preset marker.

10. The apparatus according to claim 7 , wherein the preset marker comprises a preset QR code label, and the feature point of the preset marker comprises an angular point of the preset QR code label.

11. A non-transitory computer readable medium, storing a computer program, wherein the program, when executed by a processor, causes the processor to perform operations, the operations comprising:
acquiring an image of a bucket of an excavator captured by a camera provided on an excavator body, the image of the bucket comprising a preset marker provided on the bucket;
determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker; and
converting the position and orientation information of the camera relative to the bucket into position and orientation information of the bucket relative to the excavator body.

12. The non-transitory computer readable medium according to claim 11 , wherein the three-dimensional feature information of the preset marker comprises spatial position information of a feature point of the preset marker; and
the determining position and orientation information of the camera relative to the bucket on the basis of the image of the bucket and pre-acquired three-dimensional feature information of the preset marker comprises:
determining a three-dimensional coordinate of the feature point of the preset marker according to the spatial position information of the feature point of the preset marker;
detecting the feature point of the preset marker in the image of the bucket, and determining a pixel coordinate of the feature point of the preset

Assignees

• Beijing Baidu Netcom Sci & Tec

Inventors

• Cheng Xinjing
• Yang Ruigang
• Lu Feixiang
• Yang Yajue
• Xu Hao

Classifications

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title

• G06V20/10

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V2201/07

  Target detection · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G06T7/74 Primary

  involving reference images or patches · CPC title