Systems and methods for towing vehicle and trailer with surround view imaging devices

Patent metadata

Field	Value
Publication number	US-10259390-B2
Application number	US-201715592913-A
Country	US
Kind code	B2
Filing date	May 11, 2017
Priority date	May 27, 2016
Publication date	Apr 16, 2019
Grant date	Apr 16, 2019

Abstract

Systems and method are provided for a towing vehicle towing a trailer having at least one imaging device. A method includes: receiving, from a first imaging device coupled to the trailer, a first image stream having a plurality of first images; receiving, from a second imaging device coupled to the vehicle, a second image stream having a plurality of second images; determining, at least one common feature between a first image of the first images and a second image of the second images; determining a first distance from the first imaging device to the at least one common feature and a second distance from the second imaging device to the at least one common feature; and determining, a position of the first imaging device relative to the vehicle based on the first distance, the second distance and a known position and pose of the second imaging device.

First claim

What is claimed is:

1. A method for a towing vehicle towing a trailer having at least one imaging device, comprising:
receiving, from a first imaging device coupled to the trailer, a first image stream having a plurality of first images;
receiving, from a second imaging device coupled to the towing vehicle, a second image stream having a plurality of second images;
determining, by a processor onboard the towing vehicle, at least one common feature between a first image of the plurality of first images and a second image of the plurality of second images;
determining, by the processor onboard the towing vehicle, a first distance from the first imaging device coupled to the trailer to the at least one common feature and a second distance from the second imaging device coupled to the towing vehicle to the at least one common feature; and
determining, by the processor onboard the towing vehicle, a position of the first imaging device relative to the towing vehicle based on the first distance, the second distance and a known position and pose of the second imaging device on the towing vehicle.

2. The method of claim 1 , further comprising:
determining, by the processor onboard the towing vehicle, whether the towing vehicle is in a drive range and determining, by the processor onboard the towing vehicle, whether the towing vehicle is traveling along a substantially straight path prior to determining the at least one common feature.

3. The method of claim 1 , further comprising:
determining, by the processor onboard the towing vehicle, whether the trailer is coupled to the towing vehicle; and
based on the determination, storing, by the processor onboard the towing vehicle, a calibration image from the second image stream in a datastore.

4. The method of claim 3 , further comprising:
determining, by the processor onboard the towing vehicle, whether the trailer is re-coupled to the towing vehicle;
based on the determination that the trailer is re-coupled to the towing vehicle:
retrieving the calibration image from the datastore;
acquiring a third image from the plurality of second images in the second image stream;
comparing, by the processor onboard the towing vehicle, the calibration image to the third image to determine a horizon position change between one or more pixels in the calibration image and the third image;
retrieving, by the processor onboard the towing vehicle, a pitch angle associated with the horizon position change from a tables datastore;
determining, by the processor onboard the towing vehicle, the pitch angle is less than a pitch angle threshold; and
updating the known position of the second imaging device based on the pitch angle.

5. The method of claim 1 , wherein the second imaging device includes a plurality of second cameras coupled to the towing vehicle each having a respective second image stream, the method further comprising:
processing, by the processor onboard the towing vehicle, the plurality of second images from the plurality of second image streams to determine a location of the trailer within one or more of the plurality of second images;
determining, by the processor onboard the towing vehicle, one or more feature points or patterns on a drawbar or a panel of the trailer in the one or more of the plurality of second images;
determining, by the processor onboard the towing vehicle, a third distance from one of the second cameras to the one or more feature points or patterns and a fourth distance from another one of the second cameras to the one or more feature points or patterns; and
determining, by the processor onboard the towing vehicle, a pivot angle of the trailer relative to the towing vehicle based on the third distance, the fourth distance and a known position of the plurality of second cameras on the towing vehicle.

6. The method of claim 5 , further comprising:
generating a pivot angle user interface for rendering on a display associated with the towing vehicle that graphically indicates the pivot angle of the trailer relative to the towing vehicle.

7. The method of claim 6 , further comprising:
determining, by the processor onboard the towing vehicle, that the pivot angle is greater than a pivot angle threshold; and
generating an alert user interface for rendering on the display associated with the towing vehicle based on the pivot angle as greater than the pivot angle threshold.

8. The method of claim 7 , further comprising:
generating, by the processor onboard the towing vehicle, one or more control signals to an actuator system associated with the towing vehicle to control one or more actuators of a component of the towing vehicle based on the pivot angle.

9. The method of claim 1 , wherein the second imaging device includes a plurality of second cameras coupled to the towing vehicle each having a respective second image stream, the method further comprising:
processing, by the processor onboard the towing vehicle, the plurality of second images from the plurality of second image streams to determine a location of the trailer within one or more of the plurality of second images;
determining, by the processor onboard the towing vehicle, feature points or patterns on a drawbar or a panel of the trailer in the one or more of the plurality of second images;
determining, by the processor onboard the towing vehicle, a fifth distance from one of the second cameras to the feature point or pattern and a sixth distance from another one of the second cameras to the feature point or pattern; and
determining, by the processor onboard the towing vehicle, a roll angle of the trailer relative to the towing vehicle based on the fifth distance, the sixth distance and a known position of the plurality of second cameras on the towing vehicle.

10. The method of claim 9 , further comprising:
determining, by the processor onboard the towing vehicle, that the roll angle is greater than a roll angle threshold;
generating an alert user interface for rendering on a display associated with the towing vehicle based on the roll angle as greater than the roll angle threshold; and
generating, by the processor onboard the towing vehicle, one or more control signals to an actuator system associated with the towing vehicle to control one or more actuators of a component of the towing vehicle based on the roll angle.

11. The method of claim 1 , further comprising:
determining, based on the position of the first imaging device relative to the towing vehicle and the known position and pose of the second imaging device on the towing vehicle, an area of overlap of at least one of the plurality of first images with at least one of the plurality of second images;
defining one or more boundary lines based on the area of overlap;
determining, based on a speed of the towing vehicle, a blending coefficient that defines a blending zone around the one or more boundary lines;
stitching, by the processor onboard the towing vehicle, the at least one of the plurality of first images and the at least one of the plurality of second images along the one or more boundary lines and blending the at least one of the plurality of first images and the at least one of the plurality of second images in the blending zone to generate a seamless image of a full view of a rear of the towing vehicle devoid of the trailer; and
generating, by the processor onboard the towing vehicle, a user interface that includes the full view for rendering on a display associated with the towing vehicle.

12. The method of claim 1 , further comprising:
determining, based on the position of the first imaging device relative to the towing vehicle and the known position

Assignees

• Gm Global Tech Operations Llc

Inventors

• Zhang Wende
• Wang Jinsong
• Anderson Esther

Classifications

• G06T2207/10012

  Stereo images · CPC title

• B60R2300/808

  for facilitating docking to a trailer · CPC title

• B60R2300/105

  using multiple cameras · CPC title

• B60R2300/607

  from a bird's eye viewpoint · CPC title

• B60R2300/80

  characterised by the intended use of the viewing arrangement · CPC title