Robot motion planning for photogrammetry

Patent metadata

Field	Value
Publication number	US-2019122425-A1
Application number	US-201816216476-A
Country	US
Kind code	A1
Filing date	Dec 11, 2018
Priority date	Oct 24, 2017
Publication date	Apr 25, 2019
Grant date	—

Abstract

Described herein are systems for generating 3D models using imaging data obtained using an array of light projectors, at least one object boundary detector, and a robotic member with an end effector. A first point cloud of data for an object may be generated based on boundary information obtained by the object boundary detector(s). Dimensions for the object may be determined based on the first point cloud of data. A second point cloud of data may be generated based on the dimensions for the object and a configuration of light projectors where the second point cloud corresponds to potential coordinates for a location where the robotic member and end effector can be positioned along a path around the object to capture the image data of the object. A path may be generated to avoid collision between the object and the robotic member or end effector while optimizing the number of capture location points within the second point cloud of data.

First claim

What is claimed is:

1 . A robotic gantry system for generating a 3D model of an object comprising:
a first set of light projectors configured to emit light on the object within a volume space of the robotic gantry system, the first set of light projectors associated with the robotic gantry system in a first configuration; at least one object boundary detector configured to capture boundary information about the object within the volume of space of the robotic gantry system, the at least one object boundary detector associated with the robotic gantry system in a second configuration; a robotic member configured to move within the volume of space of the robotic gantry system and comprising:
an end effector that includes at least one photogrammetry sensor and at least one depth sensor; and
a computer system configured to:
generate a first point cloud of data for the object based at least in part on the boundary information;
determine an object volume for the object that represents a relative distance from the object to the robotic gantry system and dimensions for the object based at least in part on the boundary information;
generate a second point cloud of data based at least in part on the object volume and the first configuration of the first set of light projectors, the second cloud of data corresponding to potential coordinates for a location where the robotic member and end effector can be positioned along a path around the object to capture image information for the object;
identify a first subset of data points of the second point cloud of data based at least in part on the second point cloud of data and the object volume for the object, the first subset of data points representing at least one path that would cause a collision between the robotic member or the end effector, and the object;
identify a second subset of data points of the second point cloud of data based at least in part on the second point cloud of data, the first subset of data points, and the first configuration of the first set of light projectors, wherein data points of the second subset of data points represent at least one potential path of the robotic member and the end effector that would not cause a collision between the object and the robotic member or end effector;
generate instructions to move the robotic arm in accordance with the at least one potential path within the volume of space of the robotic gantry system around the object to capture the image information for the object based at least in part on the second subset of data points; and
generate the 3D model of the object based at least in part on the image information.

2 . The robotic gantry system of claim 1 , further comprising at least one non-visible spectrum light projector, wherein the end effector either (1) further includes a non-visible spectrum light sensor or (2) the at least one photogrammetry sensor is adapted to detect presence of the non-visible spectrum light on the object.

3 . The robotic gantry system of claim 2 , wherein the computer system is further configured to generate second instructions for modulating the at least one non-visible spectrum light projector.

4 . The robotic gantry system of claim 1 , further comprising at least one non-visible spectrum light projector, and wherein the photogrammetry sensor is configured so that, when a stochastic shape distribution of a substance is present on the object, the photogrammetry sensor is adapted to detect the stochastic shape distribution while the stochastic shape distribution is exposed to non-visible spectrum light from the at least one non-visible spectrum light projector and is incapable of detecting the stochastic shape distribution whenever the stochastic shape distribution is not exposed to the non-visible spectrum light.

5 . The robotic gantry system of claim 4 , wherein the computer system is configured so that capturing the image information for the object includes capturing of a set of images that depict at least one portion of the stochastic shape distribution on the object, and generating the 3D model of the object includes generating a composite image from the set of images.

6 . The robotic gantry system of claim 1 , wherein the computer system is further configured to generate, based at least in part on the second subset of data points, first instructions for modulating the first set of light projectors to avoid occlusion by the robotic member of light projected toward the object from any light projector of the first set of light projectors during imaging of the object by the photogrammetry sensor.

7 . The robotic gantry system of claim 1 , wherein the instructions include instructions to move the robotic arm within the volume of space of the robotic gantry system around the object in a first pass to capture the image information using the photogrammetry sensor.

8 . The robotic gantry system of claim 7 , wherein the instructions further include instructions to move the robotic arm within the volume of space of the robotic gantry system around the object in a second pass to capture depth information using the at least one depth sensor of the end effector.

9 . A method of generating a 3D model, comprising:
generating a first point cloud of data for an object based at least in part on boundary information obtained by at least one object boundary detector of an image capture system; determining dimensions and a location of the object within the image capture system based at least in part on the first point cloud of data; generating a second point cloud of data based at least in part on the determined dimensions and the location of the object and a first configuration of light projectors of the image capture system, the second point cloud of data corresponding to potential coordinates for a location where a robotic member and associated end effector of the image capture system can be positioned along a path around the object to capture image information for the object; determining a first subset of data points of the second point cloud of data that represent at least one collision path that, if followed by the robotic member, would cause a collision between the object and the robotic member or the end effector, based at least in part on the second point cloud of data and the determined dimensions and the location of the object; determining, based at least in part on the first subset of data points and the first configuration of light projectors, a second subset of data points of the second point cloud of data that represent at least one potential path of the robotic member and the associated end effector of the image capture system that would not cause a collision between the object and the robotic member or the end effector when capturing the image information for the object; and generating instructions to move the robotic member and the end effector in accordance with the at least one potential path to capture the image information for the object.

10 . The method of claim 9 , further comprising generating, based at least in part on the second subset of data points, instructions for modulating the first configuration of light projectors to avoid occlusion by the robotic member of light projected toward the object from any light projector of the first configuration of light projectors.

11 . The method of claim 9 , wherein the end effector comprises at least one photogrammetry sensor and at least one depth sensor.

12 . The method of claim 9 , further comprising rotating the end effector at the location where the robotic member and end effector of the image capture system are located along a path around the object to capture the i

Assignees

• Lowes Companies Inc

Inventors

• Sheffield Mason E

Classifications

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G01B11/245 Primary

  using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence) · CPC title

• H04N23/74

  by influencing the scene brightness using illuminating means · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/661

  Transmitting camera control signals through networks, e.g. control via the Internet · CPC title