System and method for heavy equipment navigation and working edge positioning using an image acquisition device that provides distance information

Patent metadata

Field	Value
Publication number	US-9957692-B2
Application number	US-201313833697-A
Country	US
Kind code	B2
Filing date	Mar 15, 2013
Priority date	Mar 15, 2013
Publication date	May 1, 2018
Grant date	May 1, 2018

Abstract

A system and method for automatically controlling working edges on a heavy equipment vehicle is provided. A GNSS/INS system determines a location of the vehicle. A vision system calculates a location of the working edge in a vision system coordinate system by obtaining images of a target connected to the working edge using a camera with a fixed field view that obtains distance information for each pixel in the fixed field of view. The location is then transformed to the navigation system coordinate system. The transformed location information is used by a working edge control system to control the placement of the working edge.

First claim

What is claimed is:

1. A heavy equipment vehicle comprising:
a working edge for performing an operation, wherein one or more three-dimensional targets having a predefined shape are operatively connected to the working edge;
a navigation system configured to provide a location of the vehicle in a navigation system coordinate system;
a time-of-flight camera, operatively connected to the vehicle, the time-of-flight camera having a fixed field of view and configured to obtain one or more images of the one or more three-dimensional targets during the operation and to determine a distance value for each of a plurality of pixels in the image;
a vision system configured to determine location information, in a vision system coordinate system, of the working edge; and
a processor configured to translate the location information from the vision
system coordinate system to the navigation system coordinate system,
wherein the vision system is configured to determine a location of one or more three-dimensional targets using the captured one or more images, the distance values from the plurality of pixels in the one or more images, and the predefined shape of the three-dimensional targets, and to determine the location information of the working edge based on the determined location of the one or more targets.

2. The vehicle of claim 1 wherein the navigation system further comprises a GNSS sub-system operatively interconnected with an inertial navigation sub-system, wherein the GNSS sub-system and the inertial navigation sub-system share a common clock; and
wherein the captured one or more images are timestamped using the common clock.

3. The vehicle of claim 1 wherein the navigation system comprises a GNSS sub-system.

4. The vehicle of claim 1 wherein the processor translates the location information using a predefined transformation between the vision system coordinate system and the navigation system coordinate system.

5. The vehicle of claim 1 further comprising a working edge control system configured to utilize the location information in the navigation system coordinate system to control the working edge to obtain a preconfigured result.

6. The vehicle of claim 1 wherein a predefined spatial relationship exists between the target and the working edge.

7. The vehicle of claim 6 wherein the processor is further configured to perform a transformation associated with the predefined spatial relationship when transforming the location information from the vision system coordinate system to the navigation system coordinate system comprises.

8. A method for controlling a working edge on a vehicle, the method comprising:
determining a location of the vehicle in a navigation system coordinate system;
obtaining one or more images of one or more three-dimensional targets comprising a predefined shape and being operatively connected to the working edge using a time-of-flight camera that is operatively connected to the vehicle, while the working edge performs an operation, the time-of-flight camera having a fixed field of view, wherein the time-of-flight camera obtains distance information for each pixel in the fixed field of view;
determining a location of one or more three-dimensional targets using the captured one or more images, the distance values from the plurality of pixels in the one or more images, and the predefined shape of the three-dimensional targets, determining location information, in a vision system coordinate system, of the working edge based on the determined location of the one or more targets; and
transforming the location information from the vision system coordinate system to the navigation system coordinate system.

9. The method of claim 8 wherein determining the location of the vehicle in the navigation system coordinate system further comprises utilizing a global positioning sub-system and an inertial navigation sub-system to determine the location.

10. The method of claim 8 further comprising:
calculating a transformation between the navigation system coordinate system and the vision system coordinate system.

11. The method of claim 10 wherein the calculated transformation is utilized to transform the location information from the vision system coordinate system to the navigation system coordinate system.

12. The method of claim 8 further comprising using the location information in the navigation system coordinate system to operate a working edge to obtain a preconfigured result.

13. The method of claim 8 wherein a predefined spatial relationship exists between the target and the working edge.

14. A system comprising:
a vehicle having a working edge for performing an operation, wherein one or more three-dimensional targets having a predefined shape are operatively connected to the working edge, the vehicle comprising a navigation system, a time of flight camera, a vision system and a processor, wherein the vehicle is in motion and wherein the working edge is in motion;
wherein the navigation system comprises a global position sub-system and an inertial navigation sub-system, the navigation system configured to provide a location of the vehicle in a navigation system coordinate system;
wherein time of flight camera is operatively interconnected to the vehicle, has a fixed field of view, and is configured to obtain during the operation, one or more images of the one or more three-dimensional targets, using a fixed field of view and wherein the time of flight camera is configured to obtain a distance value for each pixel in each of the one or more images;
wherein the vision system is configured to determine a location of the one or more three-dimensional targets using the captured one or more images, the distance values from the plurality of pixels in the one or more images, and the predefined shape of the three-dimensional targets, and to determine location information, in a vision system coordinate system, of the working edge based on the determined location of the one or more targets; and
wherein the processor is configured to translate the location information from the vision system coordinate system to the navigation system coordinate system using a predefined transformation between the vision system coordinate system and the navigation system coordinate system and a predefined spatial relationship between the target and a predefined point on the working edge.

Assignees

• Novatel Inc
• Hexagon Technology Ct Gmbh

Inventors

• Morin Kristian

Classifications

• G05D1/0246

  using a video camera in combination with image processing means · CPC title

• E02F9/2033

  Limiting the movement of frames or implements, e.g. to avoid collision between implements and the cabin (sub-groups of E02F3/431 of E02F3/435 take precedence; for turntables E02F9/123) · CPC title

• E02F3/847 Primary

  using electromagnetic, optical or acoustic beams to determine the blade position, e.g. laser beams · CPC title

• G01C21/16

  by integrating acceleration or speed, i.e. inertial navigation · CPC title