Smart trailer hitch control using HMI assisted visual servoing

Patent metadata

Field	Value
Publication number	US-9696723-B2
Application number	US-201514747888-A
Country	US
Kind code	B2
Filing date	Jun 23, 2015
Priority date	Jun 23, 2015
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

A method for autonomously aligning a tow hitch ball on a towing vehicle and a trailer drawbar on a trailer through a human-machine interface (HMI) assisted visual servoing process. The method includes providing rearview images from a rearview camera. The method includes touching the tow ball on a display to register a location of the tow ball in the image and touching the drawbar on the display to register a location of a target where the tow ball will be properly aligned with the drawbar. The method provides a template pattern around the target on the image and autonomously moves the vehicle so that the tow ball moves towards the target. The method predicts a new location of the target as the vehicle moves and identifies the target in new images as the vehicle moves by comparing the previous template pattern with an image patch around the predicted location.

First claim

What is claimed is:

1. A method for autonomously aligning a hitch tow ball attached to a back of a towing vehicle with a drawbar attached to a trailer, said method comprising:
providing rearview images from a rearview camera on the vehicle that show the tow ball and the drawbar;
displaying the images on a touch screen display;
touching the tow ball on the display to register a position of the tow ball in the image;
touching the drawbar on the display to register a position of a target where the tow ball will be aligned with the drawbar;
providing, by a controller including a processor and memory, a template pattern around the registered target in the image;
autonomously moving the vehicle, by the controller, so that the tow ball moves towards the target;
predicting, by the controller, a position of the target in new images as the vehicle moves, using vehicle speed, vehicle steering angle and hitch trajectory;
estimating, by the controller, a new location of the target in the new images as the vehicle moves by comparing the template pattern with an image patch window provided around the predicted position of the target;
determining, by the controller, a distance between the tow ball and the new location of the target; and
stopping the vehicle, by the controller, when the distance between the tow ball and the new location of the target is less than a predetermined threshold.

2. The method according to claim 1 wherein autonomously moving the vehicle includes autonomously providing vehicle brake, throttle and steering commands.

3. The method according to claim 1 wherein autonomously moving the vehicle includes recalculating vehicle speed and steering based on the new location of the target.

4. The method according to claim 1 wherein autonomously moving the vehicle includes autonomously steering the vehicle, and wherein steering the vehicle includes converting image coordinates of the position of the tow ball and the target to world coordinates.

5. The method according to claim 4 wherein converting image coordinates of the position of the tow ball and the target to world coordinates includes using a homography matrix.

6. The method according to claim 1 wherein estimating a new location of the target in the new images includes matching the template pattern to a same pattern in the window.

7. The method according to claim 1 wherein estimating a new location of the target includes employing a Kalman-Bucy filtering process.

8. The method according to claim 7 wherein the Kalman-Bucy filtering process uses a road wheel angle input, a vehicle speed input, a Kalman gain value, and a measurement of the position of the target.

9. The method according to claim 1 further comprising providing image rectification of new images as they are obtained.

10. The method according to claim 1 further comprising initiating the alignment maneuver after the target has been registered by putting the vehicle in a reverse gear.

11. A method for autonomously aligning a hitch tow ball attached to a back of a towing vehicle with a drawbar attached to a trailer, said method comprising:
providing rearview images from a rearview camera on the vehicle that show the tow ball and the drawbar;
registering a position of the tow ball in the image;
registering a position of a target where the tow ball will be aligned with the drawbar;
autonomously moving the vehicle, by a controller including a processor and memory, so that the tow ball moves towards the target;
determining, by the controller, a distance between the tow ball and a new location of the target; and
stopping the vehicle, by the controller, when the distance between the tow ball and the new location of the target is less than a predetermined threshold.

12. The method according to claim 11 further comprising providing by the controller a template pattern around the registered target in the image, predicting by the controller a position of the target in new images as the vehicle moves using vehicle speed, vehicle steering angle and hitch trajectory, and estimating by the controller the new location of the target in the new images as the vehicle moves by comparing the template pattern with an image patch window provided around the predicted position of the target.

13. The method according to claim 12 wherein estimating the new location of the target in the new images includes matching the template pattern to a same pattern in the window.

14. The method according to claim 11 wherein autonomously moving the vehicle includes autonomously steering the vehicle, and wherein steering the vehicle includes converting image coordinates of the position of the tow ball and the target to world coordinates.

15. A system for autonomously aligning a hitch tow ball attached to a back of a towing vehicle with a drawbar attached to a trailer, said system comprising:
a rearview camera on the vehicle for providing rearview images that show the tow ball and the drawbar;
a touch screen display for displaying the images;
means for touching the tow ball on the display to register a position of the tow ball in the image;
means for touching the drawbar on the display to register a position of a target where the tow ball will be aligned with the drawbar; and
a controller including a processor and memory, said controller configured for;
providing a template pattern around the target in the image;
autonomously moving the vehicle so that the tow ball moves towards the target;
predicting a position of the target in new images as the vehicle moves;
estimating a new location of the target in the new images as the vehicle moves by comparing the template pattern with an image patch window provided around the predicted position of the target;
determining a distance between the tow ball and the new location of the target; and
stopping the vehicle when the distance between the tow ball and the new location of the target is less than a predetermined threshold.

16. The system according to claim 15 wherein the controller autonomously steers the vehicle, and wherein steering the vehicle includes converting image coordinates of the position of the tow ball and the target to world coordinates.

17. The system according to claim 16 wherein converting image coordinates of the position of the tow ball and the target to world coordinates includes using a homography matrix.

18. The system according to claim 15 wherein estimating a new location of the target employs a Kalman-Bucy filtering process.

19. The system according to claim 18 wherein the Kalman-Bucy filtering process uses a road wheel angle input, a vehicle speed input, a Kalman gain value, and a measurement of the position of the target.

Assignees

• Gm Global Tech Operations Llc

Inventors

• Zeng Shuqing
• Tong Wei
• Wang Jinsong
• Zhang Wende

Classifications

• B60W2520/10

  Longitudinal speed · CPC title

• B60R2300/202

  displaying a blind spot scene on the vehicle part responsible for the blind spot · CPC title

• B60R1/003 Primary

  for viewing trailer hitches · CPC title

• B60W2540/18

  Steering angle · CPC title

• B60W2300/14

  Tractor-trailers, i.e. combinations of a towing vehicle and one or more towed vehicles, e.g. caravans; Road trains · CPC title