Excavator bucket positioning via mobile device

Patent metadata

Field	Value
Publication number	US-10586349-B2
Application number	US-201715685617-A
Country	US
Kind code	B2
Filing date	Aug 24, 2017
Priority date	Aug 24, 2017
Publication date	Mar 10, 2020
Grant date	Mar 10, 2020

Abstract

Various embodiments provide novel tools and techniques for position and/or motion tracking, including without limitation solutions that can be used for excavation and similar applications. In a particular aspect of some embodiments, a communication device with a camera may be provided to track the position and motion of a stick of an excavator or reference features located on a stick of an excavator. A location of a bucket of an excavator may then be calculated relative to the stick or reference features on the stick of the excavator. Thus, an operator of the excavator can use the communication device to effectively and efficiently track the location of the excavator bucket without the need for a second person to provide instructions about the location of the bucket.

First claim

What is claimed is:

1. A mobile communication device, comprising:
an image sensor;
an accelerometer;
a display;
one or more processors in communication with the image sensor, the accelerometer, and the display; and
a computer readable medium in communication with the one or more processors, the computer readable medium having encoded thereon a set of instructions executable by the mobile communication device to:
capture, with the image sensor of the mobile communication device, a reference image of a plurality of reference features on a stick of an excavator while the stick of the excavator is at a known orientation;
capture input from the accelerometer of the mobile communication device;
determine an orientation of the mobile communication device, based at least in part on the input from the accelerometer of the mobile communication device;
calibrate the mobile communication device, based on orientations of the plurality of reference features in the reference image and the orientation of the mobile communication device;
capture, with the image sensor of the mobile communication device, a second image of the plurality of reference features on the stick of the excavator;
determine, from orientations of the plurality of reference features in the second image, a second orientation of the stick of the excavator; and
display, with the display of the mobile communication device, an indication of the determined second orientation of the stick of the excavator.

2. The mobile communication device of claim 1 , wherein the set of instructions are further executable by the mobile communication device to determine, from the determined second orientation of the stick of the excavator, an orientation of a bucket of the excavator, wherein displaying the indication of the determined second orientation of the stick of the excavator comprises displaying an indication of the determined orientation of the bucket of the excavator.

3. The mobile communication device of claim 1 , wherein the plurality of reference features comprise one or more reference features of one or more designated optical targets on the stick of the excavator.

4. The mobile communication device of claim 3 , wherein the set of instructions are further executable by the mobile communication device to automatically identify, without human input, depictions of the one or more designated optical targets in the reference image.

5. The mobile communication device of claim 1 , wherein the plurality of reference features comprises one or more reference features identified from a structure of the stick of the excavator.

6. The mobile communication device of claim 1 , wherein determining the second orientation of the stick of the excavator comprises determining the second orientation relative to the reference orientation of the stick of the excavator.

7. The mobile communication device of claim 6 , wherein determining the second orientation of the stick of the excavator comprises determining a reach of the stick of the excavator in the second orientation.

8. The mobile communication device of claim 6 , wherein determining the second orientation of the stick of the excavator comprises determining a depth of the stick of the excavator in the second orientation.

9. The mobile communication device of claim 1 , wherein determining the second orientation of the stick of the excavator comprises comparing orientations of the one or more reference features in the second image with orientations of the one or more reference features in the reference image.

10. The mobile communication device of claim 9 , wherein orientations of the one or more reference features comprise positions of the one or more reference features in the reference image and in the second image.

11. The mobile communication device of claim 9 , wherein orientations of the one or more reference features comprise dimensions of the one or more reference features in the reference image and in the second image.

12. The mobile communication device of claim 1 , wherein determining the second orientation of the stick of the excavator comprises determining an azimuth of the stick of the excavator in the second orientation.

13. The mobile communication device of claim 12 , wherein the set of instructions are further executable by the mobile communication device to capture, with the image sensor of the mobile communication device, a second reference image of one or more external reference features of an environment around the excavator, wherein the second image comprises the one or more external reference features, and wherein determining the azimuth of the stick of the excavator comprises comparing positions of the one or more external reference features in the second image with positions of the one or more external reference features in the second reference image.

14. The mobile communication device of claim 1 , wherein determining the second orientation of the stick of the excavator comprises determining the second orientation relative to a local or global coordinate system.

15. The mobile communication device of claim 1 , wherein the set of instructions are further executable by the mobile communication device to:
capture, with the image sensor of the mobile communication device, a second reference image of the plurality of reference features on the stick of the excavator while the stick of the excavator is at a known orientation;
calculate, based on orientations of the plurality of reference features in the reference image and the second reference image, a reference feature estimation error associated with each reference feature;
determine whether the reference feature estimation error exceeds a specified threshold;
based on a determination that the reference feature estimation error exceeds the specified threshold, capture, with the image sensor of the mobile communication device, at least one additional reference image of the plurality of reference features;
recalculate the reference feature estimation error associated with each reference feature; and
repeat the operations of capturing additional reference images of the plurality of reference features on the stick of the excavator, calculating the reference feature estimation error, and determining whether the reference feature estimation error exceeds the specified threshold until the reference feature estimation error no longer exceeds the specified threshold;
wherein calibrating the mobile communication device comprises calibrating the mobile communication device based on orientations of the plurality of reference features in at least two of the reference images.

16. The mobile communication device of claim 1 , wherein the set of instructions are further executable by the mobile communication device to:
calculate, based on the captured input from the accelerometer, an orientation estimation error associated with the orientation of the mobile communication device;
determine whether the orientation estimation error exceeds a specified threshold;
based on a determination that the orientation estimation error exceeds the specified threshold, capture additional input from the accelerometer of the mobile communication device;
recalculate the orientation estimation error based upon the additional input from the accelerometer; and
repeat the operations of capturing input from the accelerometer, calculating the orientation estimation error, and determining whether the orientation estimation error exceeds the specified threshold until the orientation estimation error no longer exceeds the specified threshold;
wherein calibrating the mobil

Assignees

• Trimble Inc

Inventors

• Li Ding
• Crozier Scott T
• Wilson Eric K

Classifications

• H04W4/026 Primary

  using orientation information, e.g. compass · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T2207/30208

  Marker matrix · CPC title

• G06T7/74

  involving reference images or patches · CPC title

• G06T2207/30164

  Workpiece; Machine component · CPC title