UAV-based sensing for worksite operations

Patent metadata

Field	Value
Publication number	US-9740208-B2
Application number	US-201514813573-A
Country	US
Kind code	B2
Filing date	Jul 30, 2015
Priority date	Jul 30, 2015
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

A sensor senses an attribute of a worksite at a location that is geographically spaced from a corresponding mobile machine. An operation is performed at the location, based upon the sensed attribute. An action signal is generated based on the effect data. An unmanned aerial vehicle communicates effect data, indicative of an effect of the operation at the location, to the mobile machine. The action signal can be used to control worksite operations.

First claim

What is claimed is:

1. A mobile machine, comprising:
a controllable mechanism that performs an operation on a worksite as the mobile machine travels over the worksite in a direction of travel;
a communication system that receives attribute data indicative of a sensed attribute of a location of the worksite, and that receives effect data indicative of an effect of the operation on the location of the worksite rearward of the mobile machine in the direction of travel, after the controllable mechanism has performed the operation at the location, the communication system receiving the effect data from a first unmanned aerial vehicle (UAV), over a communication link between the first UAV and the mobile machine; and
a control system that controls the controllable mechanism based on both the attribute data and the effect data.

2. The mobile machine of claim 1 wherein the control system further comprises:
a geographical correlation component that correlates the attribute data and the effect data to the location.

3. The mobile machine of claim 2 wherein the control system further comprises:
a prescription generator component that generates a prescribed operation for the controllable mechanism based on the attribute data, the control system controlling the controllable mechanism based on the prescribed operation.

4. The mobile machine of claim 3 wherein the control system further comprises:
a difference map generation component that determines a difference between the prescribed operation at the location and the operation performed at the location for a plurality of locations in the worksite, and generates a difference map correlating the determined differences to the plurality of locations.

5. The mobile machine of claim 4 wherein the control system generates an action signal to take a corrective action to address selected differences on the difference map.

6. The mobile machine of claim 5 wherein the control system sends the action signal to a second mobile machine to perform a follow-up operation at selected locations on the difference map.

7. The mobile machine of claim 4 wherein the control system further comprises:
a pattern identifier component that identifies a pattern of differences based on the difference map and generates a pattern signal indicative of the identified pattern.

8. The mobile machine of claim 7 wherein the control system further comprises:
a problem identifying component that receives the pattern signal and identifies a problem based on the identified pattern; and
a corrective action identifying component that identifies a corrective action based on the identified problem.

9. The mobile machine of claim 8 wherein the problem identifying component identifies machine problems with the controllable mechanism.

10. The mobile machine of claim 8 wherein the control system further comprises:
a notification generator component that generates an operator notification based on the identified problem.

11. The mobile machine of claim 4 and further comprising:
a calibration system that generates a calibration signal based on a given set of locations on the difference map, the control system calibrating at least one of the mobile machine and the controllable mechanism based on the calibration signal.

12. The mobile machine of claim 1 wherein the first UAV flies rearward of the mobile machine in the direction of travel and wherein the attribute data is received from a second UAV that flies forward of the mobile machine in the direction of travel, and further comprising:
a first physical tether that tethers the first UAV to the mobile machine.

13. The mobile machine of claim 12 and further comprising:
a second physical tether that tethers the second UAV to the mobile machine.

14. The mobile machine of claim 12 wherein the first and second UAVs each include an image capture device that captures an image of the location to generate the attribute data and the effect data, respectively.

15. A computer implemented method, comprising:
receiving attribute data indicative of a sensed attribute of a location of a worksite, forward of a mobile machine in the direction of travel, generating a prescribed operation indicator, indicative of a prescribed operation to perform at the location, based on the attribute data;
controlling, using a controller, a controllable mechanism, coupled to the mobile machine, to perform the prescribed operation at the location of the worksite, based on the prescribed operation indicator;
receiving, over a communication link, from a first unmanned aerial vehicle (UAV), effect data indicative of an effect of the operation on the location of the worksite after the controllable mechanism has performed the operation at the location; and
generating, using the controller, an action signal to perform an action based on both the attribute data and on the effect data.

16. The computer implemented method of claim 15 wherein generating a prescribed operation indicator comprises:
correlating the attribute data to the location; and
providing the location along with the prescribed operation indicator.

17. The computer implemented method of claim 16 wherein generating an action signal comprises:
determining a difference between the prescribed operation at the location and the operation performed at the location, for a plurality of locations in the worksite;
generating a difference map correlating the determined differences to the plurality of locations; and
generating the action signal to take a corrective action to address differences on the difference map.

18. The computer implemented method of claim 17 wherein generating the action signal further comprises:
identifying a pattern of differences based on the difference map;
generating a pattern signal indicative of the identified pattern;
identifying a problem based on the pattern signal;
identifying a corrective action based on the identified problem; and
generating the action signal to take the corrective action.

19. A mobile machine system, comprising:
a first unmanned aerial vehicle (UAV);
a second UAV; and
a mobile machine, comprising:
a controllable mechanism that performs an operation on a worksite, as the mobile machine moves over the worksite in a direction of travel; and
a control system that receives, from the first UAV, attribute data indicative of an attribute of the worksite sensed by the first UAV at a location forward of the mobile machine in the direction of travel, the control system generating a prescribed operation to perform at the location and controlling the controllable mechanism to perform the operation at the location based on the prescribed operation, the control system receiving effect data from the second UAV indicative of an effect of the operation performed at the location after the operation has been performed at the location, and generating an action signal based on both of the received attribute data and the effect data.

20. The mobile machine system of claim 19 wherein the first and second UAVs each have an image capture sensor that captures an image of the location, the attribute data and the effect data being indicative of the images.

Assignees

• Deere & Co

Inventors

• Sugumaran Ramanathan
• Anderson Noel W
• Hitchler Bradley J

Classifications

• B64U2101/30

  for imaging, photography or videography · CPC title

• A01B79/005

  Precision agriculture · CPC title

• G05D1/0219 Primary

  ensuring the processing of the whole working surface · CPC title

• G05D2201/0202

  Physics · mapped topic

• G05D1/0246

  using a video camera in combination with image processing means · CPC title