Method and apparatus for construction machine visualization

Patent metadata

Field	Value
Publication number	US-11086397-B2
Application number	US-201916674215-A
Country	US
Kind code	B2
Filing date	Nov 5, 2019
Priority date	Jul 3, 2014
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

A system for displaying information to an operator of a machine comprises a head tracking system and a projection system. One system for displaying information uses a projection system and a see-through display to present two-dimensional images to an operator. One system uses a projection system and a see-through display to present three-dimensional images to an operator. One system uses a pair of smart glasses to display information to a user based a direction a user is looking.

First claim

The invention claimed is:

1. A method for displaying information to an operator of a construction machine located in a cab of the construction machine, the method comprising:
determining a head position of the operator with respect to a windshield of the construction machine;
determining a direction of a gaze of the operator based on the head position of the operator with respect to the windshield;
generating an image to be displayed to the operator based on the direction of the gaze of the operator, a representation of a construction site, a desired site plan, and a construction machine operation to be performed, the image comprising an excavation area overlay, a volume overlay, an overlay depicting an image of an implement of the construction machine in a desired position, and a completion overlay; and
displaying the image to the operator.

2. The method of claim 1 , further comprising:
determining a difference between the representation of the construction site and the desired site plan; and
determining the construction machine operation to be performed based on the difference.

3. The method of claim 1 , wherein the construction machine operation to be performed comprises movement of the implement of the construction machine based on a current position of the implement and the desired position of the implement.

4. The method of claim 1 , wherein the direction of the gaze of the operator is determined further based on a position of the operator's eyes and a direction the operator's eyes are looking.

5. The method of claim 1 , wherein the generating the image to be displayed to the operator comprises:
determining a difference between the representation of the construction site and the desired site plan; and
generating a value pertaining to a percent of completion of the construction site based on the determining the difference between the representation of the construction site and the desired site plan,
wherein the completion overlay comprises the percent of completion of the construction site.

6. The method of claim 1 , further comprising:
determining that the gaze of the operator has changed;
determining a new image to be displayed to the operator based on the determining that the gaze of the operator has changed; and
displaying the new image to the operator.

7. An apparatus for displaying information to an operator of a construction machine located in a cab of the construction machine, the apparatus comprising:
a processor; and
a memory to store computer program instructions, the computer program instructions when executed on the processor, cause the processor to perform operations comprising:
determining a head position of the operator with respect to a windshield of the construction machine;
determining a direction of a gaze of the operator based on the head position of the operator with respect to the windshield;
generating an image to be displayed to the operator based on the direction of the gaze of the operator, a representation of a construction site, a desired site plan, and a construction machine operation to be performed, the image comprising an excavation area overlay, a volume overlay, an overlay depicting an image of an implement of the construction machine in a desired position, and a completion overlay; and
displaying the image to the operator.

8. The apparatus of claim 7 , the operations further comprising:
determining a difference between the representation of the construction site and the desired site plan; and
determining the construction machine operation to be performed based on the difference.

9. The apparatus of claim 7 , wherein the construction machine operation to be performed comprises movement of the implement of the construction machine based on a current position of the implement and the desired position of the implement.

10. The apparatus of claim 7 , wherein the direction of the gaze of the operator is further based on a position of the operator's eyes and a direction the operator's eyes are looking.

11. The apparatus of claim 7 , wherein the generating the image to be displayed to the operator comprises:
determining a difference between the representation of the construction site and the desired site plan; and
generating a value pertaining to a percent of completion of the construction site based on the determining the difference between the representation of the construction site and the desired site plan,
wherein the completion overlay comprises the percent of completion of the construction site.

12. The apparatus of claim 7 , the operations further comprising:
determining that the gaze of the operator has changed;
determining a new image to be displayed to the operator based on the determining that the gaze of the operator has changed; and
displaying the new image to the operator.

13. A non-transitory computer readable medium storing computer program instructions for displaying information to an operator of a construction machine located in a cab of the construction machine, which, when executed on a processor, cause the processor to perform operations comprising:
determining a head position of the operator with respect to a windshield of the construction machine;
determining a direction of a gaze of the operator based on the head position of the operator with respect to the windshield;
generating an image to be displayed to the operator based on the direction of the gaze of the operator, a representation of a construction site, a desired site plan, and a construction machine operation to be performed, the image comprising an excavation area overlay, a volume overlay, an overlay depicting an image of an implement of the construction machine in a desired position, and a completion overlay; and
displaying the image to the operator.

14. The non-transitory computer readable medium of claim 13 , the operations further comprising:
determining a difference between the representation of the construction site and the desired site plan; and
determining the construction machine operation to be performed based on the difference.

15. The non-transitory computer readable medium of claim 13 , wherein the construction machine operation to be performed comprises movement of the implement of the construction machine based on a current position of the implement and the desired position of the implement.

16. The non-transitory computer readable medium of claim 13 , wherein the direction of the gaze of the operator is further based on a position of the operator's eyes and a direction the operator's eyes are looking.

Assignees

• Topcon Positioning Systems Inc

Inventors

• Pagliani Daniele
• Finardi Nicola
• Jackson Richard
• Di Federico Ivan

Classifications

• G06V40/193

  Preprocessing; Feature extraction · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06Q10/063114

  Status monitoring or status determination for a person or group · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title