Vehicle auto-motion control system

Patent metadata

Field	Value
Publication number	US-9880560-B2
Application number	US-201314027996-A
Country	US
Kind code	B2
Filing date	Sep 16, 2013
Priority date	Sep 16, 2013
Publication date	Jan 30, 2018
Grant date	Jan 30, 2018

Abstract

An automatic control system moves a vehicle so that it can be coupled to an implement. The vehicle has propulsion and steering controls which respond to electronic input signals. A visual target is mounted on the implement. Vehicle mounted cameras generate images of the target. A motion control unit receives and processes the images, and generates vehicle movement commands as a function of the images which are communicated to the propulsion and steering controls. The motion control unit includes a perception engine connected to the cameras, a controller connected to the perception engine, and a vehicle/operator interface which is connected to the controller, and to the propulsion and steering controls. The perception engine captures images from the cameras, un-distorts the images, and searches for the target. If the target is found, the perception engine recovers vehicle posture information and transmits said information to the controller.

First claim

We claim:

1. An automatic control system for moving a vehicle with respect to an implement to be attached to the vehicle, the vehicle having propulsion and steering controls which respond to electronic input signals and the implement including a visual target mounted to the implement, the automatic control system comprising:
a manual movement control;
an operator interface;
a camera mounted on the vehicle, the camera generating images; and
a motion control unit which receives the images from the camera, the motion control unit configured to
generate vehicle movement and command signals based on user input through the manual movement control while operating in a manual machine operation mode,
process said images generated by the camera,
generate a prompt on the operator interface in response to detecting the visual target in an image from the camera while operating in the manual machine operation mode, the prompt instructing an operator to actuate automated hitching,
apply a perception engine and automatically generate vehicle movement command signals as a function of vehicle posture information determined based at least in part on the images only when automated hitching is actuated by the operator in response to the generated prompt on the operator interface, and
communicate the command signals to the propulsion and steering controls, wherein, in response to receiving the command signals, the propulsion and steering controls move the vehicle to a coupling position so that the implement can be coupled to the vehicle,

wherein the motion control unit is configured to process said images by applying the perception engine that
determines a region of interest in an image generated by the camera and searches for the visual target within the region of interest;
when the visual target is found in the image, determines the vehicle posture information based on a location of the visual target in the image and updates the region of interest, wherein the updated region of interest is larger than the visual target in the image and smaller than an entire image from the camera; and
after the visual target is found, continues to process subsequent images from the camera by searching for the visual target only within a region of interest updated based on a location of the visual target in a previous image, and determining vehicle posture information based on a detected location of the visual target in each image,
wherein a field of view of the region of interest relative to the entire image from the camera is determined based on the field of view of the region of interest for the previous image and increases as the motion control unit causes the vehicle to move closer to the coupling position.

2. The control system of claim 1 , wherein:
the perception engine updates the region of interest to be approximately 120% larger than the target in each direction.

3. The control system of claim 2 , wherein:
the target displays a chessboard pattern.

4. The control system of claim 3 , wherein the camera includes a near-view camera and a far-view camera.

5. A coupling alignment system for moving a vehicle into a coupling position with respect to an implement to be attached to the vehicle, the coupling alignment system comprising an electronic control unit configured to
receive a first image from a camera positioned on the vehicle,
search the first image for a visual target within a defined region of interest for the camera, the defined region of interest including a field of view that is larger than the visual target in the first image and smaller than an entire area of the first image, the visual target being mounted to the implement,
generate vehicle movement command signals based at least in part on the location of the visual target in the first image, wherein the vehicle movement command signals cause the vehicle to move relative to the visual target when received by one or more vehicle actuators,
update the defined region of interest for the camera based on a detected location of the visual target in the first image and based on a size of the field of view of the defined region of interest used for the first image such that a size of a field of view of the updated defined region of interest increases as the vehicle is moved closer to the visual target,
receive a second image from the camera after the vehicle has moved relative to the visual target, and
search the second image for the visual target only within the updated defined region of interest for the camera.

6. An automatic control system for moving a vehicle with respect to an implement to be attached to the vehicle, the vehicle having propulsion and steering controls which respond to electronic input signals and the implement including a visual target mounted to the implement, the automatic control system comprising:
a camera mounted on the vehicle, the camera generating images; and
a motion control unit which receives the images from the camera, the motion control unit configured to
process said images,
generate vehicle movement command signals as a function of vehicle posture information determined based at least in part on the images, and
communicate the command signals to the propulsion and steering controls, wherein, in response to receiving the command signals, the propulsion and steering controls move the vehicle to a coupling position so that the implement can be coupled to the vehicle,

wherein the motion control unit is configured to process said images by applying a perception engine that
determines a region of interest in an image generated by the camera and searches for the visual target within the region of interest;
when the visual target is found in the image, determines the vehicle posture information based on a location of the visual target in the image and updates the region of interest, wherein the updated region of interest is larger than the visual target in the image and smaller than an entire image from the camera; and
after the visual target is found, continues to process subsequent images from the camera by searching for the visual target only within a region of interest updated based on a location of the visual target in a previous image, and determining vehicle posture information based on a detected location of the visual target in each image,
wherein a field of view of the region of interest relative to the entire image from the camera is determined based on the field of view of the region of interest for the previous image and increases as the motion control unit causes the vehicle to move closer to the coupling position,

wherein the perception engine updates the region of interest by determining a field of view for the updated region of interest based on the field of view for the region of interest used for the previous image, wherein
X min _ new =X min _ old −dX;
X max _ new =X max _ old +dX;
Y min _ new =Y min _ old −dY;
Y max _ new =X max _ old +dY;
dX=c * ( X max —old −X min —old );
dY=c * ( Y max —old −Y min —old );

wherein X max _ old and X min _ old define the field of view on an X-axis for the region of interest used for the previous image,
wherein Y max _ old and Y min _ old define the field of view on a Y-axis for the region of interest used for the previous image,
wherein X max _ new and X min _ new define the field of view on an X-axis for the updated region of interest,
wherein Y max _ new and Y min _ new define the field of view on a Y-axis for the updated region of interest, and
wherein c is a calibration constant.

7. The automatic control system of claim 6 , further

Assignees

• Deere & Co

Inventors

• Han Shufeng
• Turner Christopher D
• Buerkle Bryan K
• Duppong Steven A

Classifications

• B60D1/36

  for facilitating connection, e.g. hitch catchers · CPC title

• A01B71/063

  for enabling transmission coupling and connection between tractor and implements to be controlled from the driver's seat · CPC title

• A01B69/00

  Steering of agricultural machines or implements; Guiding agricultural machines or implements on a desired track · CPC title

• A01B59/062 Primary

  the connection comprising a rigid interface frame on the tractor · CPC title

• B60D1/62

  involving supply lines, electric circuits or the like · CPC title