Automatically scanning and representing an environment having a plurality of features

Patent metadata

Field	Value
Publication number	US-10349035-B2
Application number	US-201514941826-A
Country	US
Kind code	B2
Filing date	Nov 16, 2015
Priority date	Nov 16, 2015
Publication date	Jul 9, 2019
Grant date	Jul 9, 2019

Abstract

Automatic scanning and representing an environment having a plurality of features, for example, includes scanning the environment along a scanning path, interspersing a plurality of localized scanning of the plurality of features in the environment during the scanning along the scanning path of the environment wherein the interspersed localized scanning of the plurality of features in the environment being different from the scanning the environment along the scanning path, and obtaining a representation of at least a portion of the environment based on the scanning of the environment and the interspersed localized scanning of the plurality of features in the environment.

First claim

The invention claimed is:

1. A computer-implemented method comprising:
scanning an environment along a scanning path;
interspersing a plurality of localized scanning of a plurality of candidate features in the environment during the scanning along the scanning path of the environment, the interspersed localized scanning of the plurality of candidate features in the environment being different from the scanning the environment along the scanning path; and
obtaining a representation of at least a portion of the environment based on the scanning of the environment and the interspersed localized scanning of the plurality of candidate features in the environment,
wherein interspersing the plurality of localized scanning of the plurality of candidate features includes:
generating a set of data corresponding to one candidate feature of the plurality of candidate features by scanning one candidate of the plurality of candidate features a plurality of times,
determining the one candidate of the plurality of candidate features exceeds a certainty threshold using the set of data,
recording the candidate location, a robot location, and a robot end effector orientation in response to the determining, and
updating a robot forward kinematic model for the robot end effector using the recorded robot location and robot end effector orientation.

2. The computer-implemented method of claim 1 wherein the interspersing the plurality of localized scanning of the plurality of candidate features in the environment comprises identifying one of the plurality of candidate features during the scanning of the scanning path of a first portion of the environment, scanning a localized portion of the environment having the identified one of the plurality of candidate features in the first portion of the environment, continuing the scanning along the scanning path of the environment another portion of the environment, identifying another one of the plurality of candidate features during the scanning of the scanning path of the another portion of the environment, and scanning another localized portion of the environment having the identified another of the plurality of candidate features in the another portion of the environment.

3. The computer-implemented method of claim 1 wherein the scanning along the scan path of the environment is at a first resolution or speed and the scanning of the localized portions of the environment is at a second resolution or speed different from the first resolution or speed.

4. The computer-implemented method of claim 1 wherein the scanning of the environment along the scanning path is performed using a first scanning parameter, the interspersing comprises using at least one criteria for identifying the plurality of candidate features along the first scanning path, and the scanning of the localized portions is based on the plurality of identified candidate features and performed using a second scanning parameter different from the first scanning parameter.

5. The computer-implemented method of claim 1 wherein the scanning along the scanning path employs a 3D imager and the scanning the localized portions employs the 3D imager.

6. The computer-implemented method of claim 1 wherein the obtaining the representation comprises obtaining a first plurality of frames of data of the scanning along the scan path of the environment and a plurality of frames of data of the localized portions of the plurality of candidate features of the environment, and merging together data from the first plurality of frames of data of the scanning along the scan path of the environment and the plurality of frames of data of the localized portions of the plurality of candidate features of the environment.

7. The computer-implemented method of claim 1 wherein the scanning the environment along the scanning path comprises updating the statistical model of feature location prediction based on prior feature detection results, and estimating a certainty of a current detected feature using the updated statistical model of feature location prediction and a current feature detection result, and wherein the obtaining the representation of the interspersed localized scanning of the plurality of candidate features in the environment is based on the estimated certainty of the current detected feature being above a threshold.

8. The computer-implemented method of claim 1 wherein the scanning along the scan path comprises scan across a plane.

9. The computer-implemented method of claim 1 wherein the scanning the localized portions comprises scanning the localized portions from a plurality of orientations.

10. The computer-implemented method of claim 1 wherein the scanning along a scan path and the scanning of the localized portions comprises employing a robotically supported movable imager.

11. The computer-implemented method of claim 1 wherein the environment comprises a rock wall and the plurality of candidate features comprise a plurality of elongated holes defining axes for receiving charges for mining.

12. The computer-implemented method of claim 11 wherein the scanning the wall along the scanning path comprises updating the statistical model of feature location prediction and a drilling map matching based on prior hole detection results, and estimating a certainty of a current detected hole using the updated statistical model of feature location prediction, the updated drilling map matching, and a current hole detection result, and wherein the obtaining the representation of the interspersed localized scanning of the plurality of holes in the wall is based on the estimated certainty of the current detected feature being above a threshold.

13. The computer-implemented method of claim 11 further comprising determining orientations of the axes of the plurality of holes based on the scanning of the localized portions.

14. A system comprising:
a memory; and
a physical processor in communications with the memory,

wherein the system is configured to perform a method, said method comprising:
scanning an environment along a scanning path;
interspersing a plurality of localized scanning of a plurality of candidate features in the environment during the scanning along the scanning path of the environment, the interspersed localized scanning of the plurality of candidate features in the environment being different from the scanning the environment along the scanning path; and
obtaining a representation of at least a portion of the environment based on the scanning of the environment and the interspersed localized scanning of the plurality of candidate features in the environment,
wherein interspersing the plurality of localized scanning of the plurality of candidate features includes:
generating a set of data corresponding to one candidate feature of the plurality of candidate features by rescanning around one candidate of the plurality of candidate features,
determining the one candidate of the plurality of candidate features exceeds a certainty threshold using the set of data,
recording the candidate location, a robot location, and a robot end effector orientation in response to the determining, and
updating a robot forward kinematic model for the robot end effector using the recorded robot location and robot end effector orientation.

15. A non-transitory computer readable storage medium readable by a processor comprising:
instructions stored on the non-transitory computer readable storage medium for execution by the processor for performing a method comprising:
scanning an environment along a scanning path;
interspersing a pluralit

Assignees

• Abb Schweiz Ag

Inventors

• Wang Jianjun
• Zhang Biao
• Martinez Carlos
• Morato Carlos W
• Boca Remus

Classifications

• H04N23/60

  Control of cameras or camera modules · CPC title

• G06T7/75 Primary

  involving models · CPC title

• H04N13/204 Primary

  using stereoscopic image cameras (stereoscopic photography G03B35/00) · CPC title

• H04N5/232

  Electricity · mapped topic

• G06T2207/10016

  Video; Image sequence · CPC title