Three-dimensional reconstruction method and apparatus for material pile, electronic device, and computer-readable medium

Patent metadata

Field	Value
Publication number	US-2020242829-A1
Application number	US-201916676281-A
Country	US
Kind code	A1
Filing date	Nov 6, 2019
Priority date	Jan 25, 2019
Publication date	Jul 30, 2020
Grant date	—

Abstract

Embodiments of the present disclosure relate to a three-dimensional reconstruction method and apparatus for a material pile, an electronic device, and a computer-readable medium. The method may include: acquiring, in response to an instruction for controlling an excavator body of an excavator to rotate to transport materials being detected, a sequence of depth images of an excavated material pile collected by a binocular camera provided on a side of the excavator; and performing three-dimensional reconstruction based on the sequence of depth images of the material pile, to generate a three-dimensional model of the material pile.

First claim

What is claimed is:

1 . A three-dimensional reconstruction method for a material pile, comprising:
acquiring, in response to an instruction for controlling an excavator body of an excavator to rotate to transport materials being detected, a sequence of depth images of the material pile collected by a binocular camera provided on a side of the excavator; and performing three-dimensional reconstruction based on the sequence of depth images of the material pile, to generate a three-dimensional model of the material pile.

2 . The method according to claim 1 , wherein the acquiring, in response to an instruction for controlling an excavator body to rotate to transport materials being detected, a sequence of depth images of the material pile collected by a binocular camera provided on a side of the excavator comprises:
turning on, in response to the instruction for controlling the excavator body to rotate to transport materials being detected, the binocular camera provided on the side of the excavator; and acquiring the sequence of depth images of the material pile, the sequence of depth images of the material pile being collected by the binocular camera during the rotation of the excavator body to transport materials.

3 . The method according to claim 2 , wherein binocular cameras are provided on both sides of the excavator;
the turning on, in response to the instruction for controlling the excavator body to rotate to transport materials being detected, the binocular camera provided on the side of the excavator comprises: determining, in response to the instruction for controlling the excavator body to rotate to transport materials being detected, a binocular camera provided on the side turned to the material pile according to the rotation direction of the excavator body, and issuing a turn-on instruction to the binocular camera on the side turned to the material pile.

4 . The method according to claim 2 , wherein the method further comprises:
turning off, in response to an instruction for controlling a bucket of the excavator to unload materials being detected, the binocular camera provided on the side of the excavator.

5 . The method according to claim 1 , wherein the performing three-dimensional reconstruction based on the sequence of depth images of the material pile, to generate a three-dimensional model of the material pile comprises:
converting the sequence of depth images of the material pile into a sequence of three-dimensional point clouds; fusing the sequence of three-dimensional point clouds based on overlaps of adjacent images in the sequence of depth images, to obtain a fused three-dimensional point cloud of the material pile; and generating the three-dimensional model of the material pile based on the three-dimensional point cloud of the material pile.

6 . The method according to claim 1 , wherein the method further comprises:
planning, according to the three-dimensional model of the material pile, an excavation operation for excavating the material pile, and generating an instruction for excavating.

7 . The method according to claim 6 , wherein excavation operation for excavating the material pile includes at least one of:
an excavating point, an excavating angle and an excavating trajectory.

8 . A three-dimensional reconstruction apparatus for a material pile, comprising:
at least one processor; and a memory storing instructions, the instructions when executed by the at least one processor, cause the at least one processor to perform operations, the operations comprising: acquiring, in response to an instruction for controlling an excavator body of an excavator to rotate to transport materials being detected, a sequence of depth images of the material pile collected by a binocular camera provided on a side of the excavator; and performing three-dimensional reconstruction based on the sequence of depth images of the material pile to, generate a three-dimensional model of the material pile.

9 . The apparatus according to claim 8 , wherein the acquiring, in response to an instruction for controlling an excavator body of the excavator to rotate to transport materials being detected, a sequence of depth images of the material pile collected by the binocular camera provided on the side of the excavator by the following way:
turning on, in response to the instruction for controlling the excavator body to rotate to transport materials being detected, the binocular camera provided on the side of the excavator; and acquiring the sequence of depth images of the material pile, the sequence of depth images of the material pile being collected by the binocular camera during the rotation of the excavator body to transport materials.

10 . The apparatus according to claim 9 , wherein binocular cameras are provided on both sides of the excavator;
the turning on, in response to the instruction for controlling the excavator body to rotate to transport materials being detected, the binocular camera provided on the side of the excavator comprises: determining, in response to the instruction for controlling the excavator body to rotate to transport materials being detected, a binocular camera provided on the side turned to the material pile according to the rotation direction of the excavator body, and issuing a turn-on instruction to the binocular camera on the side turned to the material pile.

11 . The apparatus according to claim 9 , wherein the operations further comprise:
turning off, in response to an instruction for controlling a bucket of the excavator to unload materials being detected, the binocular camera provided on the side of the excavator.

12 . The apparatus according to claim 8 , wherein the performing three-dimensional reconstruction based on the sequence of depth images of the material pile, to generate a three-dimensional model of the material pile comprises:
converting the sequence of depth images of the material pile into a sequence of three-dimensional point clouds; fusing the sequence of three-dimensional point clouds based on overlaps of adjacent images in the sequence of depth images, to obtain a fused three-dimensional point cloud of the material pile; and generating the three-dimensional model of the material pile based on the three-dimensional point cloud of the material pile.

13 . The apparatus according to claim 8 , wherein the operations further comprise:
planning, according to the three-dimensional model of the material pile, an excavation operation for excavating the material pile, and generate an instruction for excavating.

14 . A computer-readable medium, storing a computer program thereon, wherein the program, when executed by a processor, causes the processor to perform operations, the operations comprising:
acquiring, in response to an instruction for controlling an excavator body of an excavator to rotate to transport materials being detected, a sequence of depth images of the material pile collected by a binocular camera provided on a side of the excavator; and performing three-dimensional reconstruction based on the sequence of depth images of the material pile, to generate a three-dimensional model of the material pile.

15 . The medium according to claim 14 , wherein the acquiring, in response to an instruction for controlling an excavator body to rotate to transport materials being detected, a sequence of depth images of the material pile collected by a binocular camera provided on a side of the excavator comprises:
turning on, in response to the instruction for controlling the excavator body to rotate to transport mate

Assignees

• Beijing Baidu Netcom Sci & Tec

Inventors

• Cheng Xinjing
• Yang Ruigang
• Lu Feixiang
• Yang Yajue
• Xu Hao

Classifications

• E02F9/261

  Surveying the work-site to be treated · CPC title

• E02F9/205

  Remotely operated machines, e.g. unmanned vehicles (E02F3/8866 takes precedence) · CPC title

• G06T2211/416

  Exact reconstruction · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T17/20 Primary

  Finite element generation, e.g. wire-frame surface description, {tesselation} · CPC title