Visual assistance and control system for a work machine

Patent metadata

Field	Value
Publication number	US-10829911-B2
Application number	US-201816122121-A
Country	US
Kind code	B2
Filing date	Sep 5, 2018
Priority date	Sep 5, 2018
Publication date	Nov 10, 2020
Grant date	Nov 10, 2020

Abstract

A work machine includes a frame and a set of ground engaging elements movably supported by the frame and driven by an engine to drive movement of the machine in a worksite. The machine includes a location sensor configured to generate a machine location sensor signal indicative of a location of the machine. The machine includes a communication component that communicates with a worksite server and retrieves object location data. The machine also includes virtual model generator logic that determines the object is within a field of view of an operator of the machine, based on the machine location sensor signal and object location data, and generates an augmentation indication indicative of the determination. The machine includes augmentation logic that generates and displays an augmented reality overlay, based on the augmentation indication, and displays an indication of the object proximate the object within the field of view of the operator.

First claim

What is claimed is:

1. A mobile work machine, comprising:
a frame;
a set of ground engaging elements movably supported by the frame;
an actuator coupled to the set of ground engaging elements to controllably drive movement of the set of ground engaging elements to drive movement of the mobile work machine in a worksite;
a location sensor configured to sense a location of the mobile work machine and generate a machine location sensor signal indicative of a location of the mobile work machine;
field of view logic that receives object location data of an object wherein the object comprises a portion of a surface, and determines whether the object is within a field of view of an operator of the mobile work machine, based on the machine location sensor signal and object location data, and generates a field of view indication indicative of the determination;
terrain generator logic that compares a height of the portion of the surface to a threshold elevation and generates an elevation indication indicating an elevation of the portion of the surface relative to the threshold elevation based on the determination;
a display device;
augmentation logic that generates an augmented display, based on the field of view indication, and controls the display device to display the augmented display showing the elevation indication at a location proximate the portion of the surface, in perspective, within the field of view of the operator; and
wherein the elevation indication comprises:
a first indication, if the elevation indication is indicative of the portion of the surface being below the threshold elevation; and
a second indication visually different from the first indication, if the elevation indication is indicative of the portion of the surface being above the threshold elevation.

2. The work machine of claim 1 , wherein the first indication comprises a first color, and the second indication comprises a second color.

3. The work machine of claim 1 , wherein the first indication comprises a first visual pattern, and the second indication comprises a second visual pattern.

4. The work machine of claim 1 , wherein the augmentation logic displays the indication of the object as a third indication, visually different from the first and second indications, if the elevation indication is indicative of the portion of the surface being substantially at the threshold elevation.

5. The work machine of claim 1 field of view logic that receives second object location data indicative of location of a second object wherein the second object comprises a hazard and wherein the work machine further comprises:
hazard generator logic that determines a type of hazard and generates a hazard indication based on the determination; and
wherein the augmentation logic displays the hazard indication indicative of a hazard location.

6. The work machine of claim 5 , wherein the hazard comprises a mobile object and the indication of the object visually distinguishes the hazard from its surrounding.

7. The work machine of claim 5 , wherein the hazard comprises a utility and wherein the augmentation logic is configured to generate the indication of the object to identify a type of the utility.

8. The work machine of claim 5 , wherein the augmentation logic is configured to generate the indication of the object to include an indication of a distance to the object from the mobile work machine.

9. The work machine of claim 1 , further comprising:
non-visual alert generator logic configured to generate an audible alert when the work machine is within a threshold distance of the object.

10. The work machine of claim 5 , further comprising:
an operator sensor that detects a characteristic of the operator and generates an operator sensor signal indicative of the sensed characteristic; and
wherein the field of view logic determines whether the object is within a field of view of an operator of the mobile work machine, based on the operator sensor signal.

11. The work machine of claim 1 further comprising:
collision logic configured to receive a commanded movement from the operator and determine whether the commanded movement will result in contact between the work machine and a second object in the worksite.

12. The work machine of claim 1 wherein the display device comprises ahead-mounted device.

13. The work machine of claim 1 wherein the display device comprises a cab window.

Assignees

• Deere & Co

Inventors

• Kennedy Thomas L
• Myers David J

Classifications

• G06V20/17

  taken from planes or by drones · CPC title

• G06V20/38

  Outdoor scenes · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G02B27/017

  Head mounted · CPC title

• E02F9/264 Primary

  Sensors and their calibration for indicating the position of the work tool · CPC title