Anticipatory modification of machine settings based on predicted operational state transition

Patent metadata

Field	Value
Publication number	US-11697917-B2
Application number	US-201916523550-A
Country	US
Kind code	B2
Filing date	Jul 26, 2019
Priority date	Jul 26, 2019
Publication date	Jul 11, 2023
Grant date	Jul 11, 2023

Abstract

Methods and systems for adjusting operating parameters of a machine in anticipation of a transition from a current operational state to a predicted subsequent operational state. An electronic controller receives a data stream indicative of actuator settings, sensor outputs, and/or operator control settings and applies a pattern detection AI that is configured to determine a current operational state of the machine based on patterns detected in the data stream. The controller then applies a reinforcement learning AI that is configured to produce as an output one or more target operating parameters based at least in part on a predicted subsequent operational state of the machine. The one or more target operating parameters are applied to the machine and at least one performance metric of the machine is monitored. The reinforcement learning Ai is retrained based at least in part on the monitored performance metric(s).

First claim

What is claimed is:

1. A method for adjusting operating parameters of a machine in anticipation of a transition from a current operational state to a subsequent operational state, wherein the machine includes an excavator with a boom arm and a bucket coupled to a distal end of the boom arm, the method comprising:
receiving, by an electronic controller, a data stream indicative of at least one selected from a group consisting of actuator settings, sensor outputs, and operator control settings;
applying, by the electronic controller, a pattern detection AI, wherein the pattern detection AI is configured to
receive the data stream as an input,
determine a current operational state of the machine based on patterns detected in the data stream, and
determine, based on the data stream, when the current operational state of the machine is a lifting of the bucket;

applying, by the electronic controller, a reinforcement learning AI, wherein the reinforcement learning AI is configured to
identify the predicted subsequent operational state by identifying a current operation of the machine based on the determined current operational state of the machine and a sequence of previously determined operational states of the machine, wherein a current operation of the machine is defined based on a sequence of operational states where the machine repeats the operational states of lifting the material at a first location, traveling to a second location, releasing the material at the second location, and returning to the first location, and wherein the current operation is a lift-and-carry operation, and
produce as an output one or more target operating parameters based at least in part on the predicted subsequent operational state, wherein the output is an adjustment to a power distribution of the machine to provide more power to the drivetrain in anticipation of the machine transitioning from the lifting operational state to a driving operational state in the lift-and-carrying operation;

applying, by the electronic controller, the one or more target operating parameters to the machine;
monitoring at least one performance metric of the machine after application of the one or more target operating parameters; and
retraining the reinforcement learning AI based on the at least one monitored performance metric.

2. The method of claim 1 , further comprising using unsupervised learning to retrain the pattern detection AI based on the data stream.

3. The method of claim 1 , wherein the data stream includes a time-domain sequence of sensor outputs indicative of an operating characteristic of the machine.

4. The method of claim 3 , wherein the machine includes an excavator arm, and wherein the data stream includes a time-domain sequence of sensor outputs indicative of a pose of the excavator arm.

5. The method of claim 1 , wherein receiving the data stream includes receiving a time-domain data stream and converting the time-domain data stream to a frequency-domain data stream, and wherein the pattern detection AI is configured to receive the data stream as an input by receiving the frequency-domain data stream as an input.

6. The method of claim 1 , wherein the pattern detection AI is configured to receive as inputs the data stream and a sequence of previously detected operational states of the machine and is configured to produce as an output an identification of the predicted subsequent operational state of the machine, and wherein the reinforcement learning AI is configured to receive as inputs the data stream and the identification of the predicted subsequent operational state of the machine.

7. The method of claim 6 , wherein the pattern detection AI is further configured to produce as another output a predicted timing of a transition from the current operational state to the predicted subsequent operational state, and wherein the reinforcement learning AI is configured to receive as another input the predicted timing of the transition.

8. The method of claim 1 , wherein the pattern detection AI is configured to produce as an output an identification of the current operational state of the machine, and wherein the reinforcement learning AI is configured to receive as inputs the identification of the current operational state of the machine and a sequence of previously detected operational states of the machine.

9. The method of claim 8 , wherein the reinforcement learning AI is further configured to produce as an output an identification of the predicted subsequent operational state of the machine.

10. The method of claim 9 , wherein the reinforcement learning AI is further configured to produce as an output a predicted timing of a transition from the current operational state to the predicted subsequent operational state of the machine, and wherein applying the one or more target operating parameters to the machine includes applying the one or more target operating parameters to the machine based on the predicting timing of the transition.

11. The method of claim 1 , wherein applying the one or more target operating parameters to the machine includes applying the one or more target operating parameters to the machine in real-time as the one or more target operating parameters are output by the reinforcement learning AI.

12. The method of claim 1 , wherein the one or more target operating parameters includes an adjustment to at least one actuator of the machine.

13. The method of claim 12 , wherein the adjustment to the at least one actuator of the machine includes an increase in the hydraulic pressure produced by a hydraulic pump that will be used to extend one or more hydraulic cylinders after the transition from the current operational state to the predicted subsequent operational state.

14. The method of claim 1 , wherein the one or more target operating parameters includes an adjustment to a machine setting of the machine.

15. A method for adjusting operating parameters of a machine in anticipation of a transition from a current operational state to a subsequent operational state, the method comprising:
receiving, by an electronic controller, a data stream indicative of at least one selected from a group consisting of actuator settings, sensor outputs, and operator control settings;
applying, by the electronic controller, a pattern detection AI, wherein the pattern detection AI is configured to receive the data stream as an input and to determine a current operational state of the machine based on patterns detected in the data stream;
applying, by the electronic controller, a reinforcement learning AI, wherein the reinforcement learning AI is configured to produce as an output one or more target operating parameters based at least in part on a predicted subsequent operational state, wherein the one or more target operating parameters includes an adjustment to a machine setting of the machine;
applying, by the electronic controller, the one or more target operating parameters to the machine, wherein the adjustment to the machine setting includes an adjustment to a power distribution of the machine to provide more power to a hydraulic system of the machine in anticipation of a transition to a predicted subsequent operational state in which the hydraulic system will utilize more power than in the current operational state of the machine;
monitoring at least one performance metric of the machine after application of the one or more target operating parameters; and
retraining the reinforcement learning AI based on the at least one monitored performance metric.

Assignees

• Deere & Co

Inventors

• Myers David J
• Lehmann Doug M
• Hamilton Robert A
• Campbell Kevin W
• Keene Joseph R

Classifications

• G06N20/00

  Machine learning · CPC title

• E02F3/43

  Control of dipper or bucket position; Control of sequence of drive operations · CPC title

• E02F9/2025 Primary

  Particular purposes of control systems not otherwise provided for (E02F3/16, E02F3/26, sub-groups of E02F3/43, E02F3/651, sub-groups of E02F3/84, E02F3/907, E02F5/145 take precedence) · CPC title

• G05B13/02 Primary

  electric · CPC title

• E02F9/265 Primary

  with follow-up actions (e.g. control signals sent to actuate the work tool) · CPC title