Systems and methods for detecting trailer angle

Patent metadata

Field	Value
Publication number	US-11200430-B2
Application number	US-201816181020-A
Country	US
Kind code	B2
Filing date	Nov 5, 2018
Priority date	Nov 5, 2018
Publication date	Dec 14, 2021
Grant date	Dec 14, 2021

Abstract

Systems and methods for detecting trailer angle are provided. In one aspect, an in-vehicle control system includes an optical sensor configured to be mounted on a tractor so as to face a trailer coupled to the tractor, the optical sensor further configured to generate optical data indicative of an angle formed between the trailer and the tractor. The system further includes a processor and a computer-readable memory in communication with the processor and having stored thereon computer-executable instructions to cause the processor to receive the optical data from the optical sensor, determine at least one candidate plane representative of a surface of the trailer visible in the optical data based on the optical data, and determine an angle between the trailer and the tractor based on the at least one candidate plane.

First claim

What is claimed is:

1. An in-vehicle control system for a tractor and trailer, comprising:
an optical sensor configured to be mounted on the tractor and generate optical data indicative of an angle formed between the trailer and the tractor;
a processor; and
a computer-readable memory in communication with the processor and having stored thereon computer-executable instructions to cause the processor to:
receive the optical data, the optical data comprising point cloud data,
detect one or more markers on a surface of the trailer via identifying points from the point cloud data having a reflectance value lower than a threshold reflectance value,
determine at least one candidate plane representative of the surface of the trailer based on the point cloud data, the candidate plane being visible in the optical data, and
determine the angle between the trailer and the tractor based at least on the at least one candidate plane and the one or more markers.

2. The system of claim 1 , wherein the optical sensor is positioned so as to face the trailer.

3. The system of claim 1 , wherein the optical sensor comprises a LiDAR detector configured to generate the optical data indicative of a spatial position of a visible surface of the trailer.

4. The system of claim 1 , wherein the optical sensor comprises a camera configured to generate an image comprising a representation of the one or more markers on the surface of the trailer.

5. The system of claim 4 , wherein the one or more markers comprise a quick response (QR) code.

6. The system of claim 1 , wherein:
the points of the point cloud data are representative of the location of the surface of the trailer in space, and
the memory further has stored thereon computer-executable instructions to cause the processor to:
remove points belonging to edge areas of the trailer from the identified points to define a remaining subset of identified points,
wherein determining the at least one candidate plane is performed based on the remaining subset of identified points.

7. The system of claim 1 , wherein the memory further has stored thereon computer-executable instructions to cause the processor to:
refine the at least one candidate plane using an expectation-maximization framework comprising a plurality of iterations of: i) an expectation step comprising determining a mapping function and ii) a maximization step comprising optimizing an objective function.

8. The system of claim 1 , wherein the memory further has stored thereon computer-executable instructions to cause the processor to:
determine that a set of the identified points are within a first threshold distance from each other, and
group the set of identified points into a cluster corresponding to one of the one or more markers.

9. The system of claim 8 , wherein the memory further has stored thereon computer-executable instructions to cause the processor to:
determine a shape feature from the cluster, and
assign a marker identifier to the cluster based on the shape feature.

10. The system of claim 9 , wherein the memory further has stored thereon computer-executable instructions to cause the processor to:
determine that a distance from the at least one candidate plane to the marker corresponding to the marker identifier is less than a second threshold distance, and
determine that the at least one candidate plane belongs to the surface of the trailer in response to determining that the distance from the at least one candidate plane to the marker is less than the second threshold distance,
wherein the determining of the angle between the trailer and the tractor is performed in response to determining that the at least one candidate plane belongs to the surface of the trailer.

11. A non-transitory computer readable storage medium having stored thereon instructions that, when executed, cause at least one computing device to:
receive optical data from an optical sensor, the optical data comprising point cloud data, the optical sensor configured to be mounted on a tractor and generate the optical data indicative of an angle formed between a trailer and the tractor;
detect one or more markers on a surface of the trailer via identifying points from the point cloud data having a reflectance value lower than a threshold reflectance value;
determine at least one candidate plane representative of the surface of the trailer based on the point cloud data, the candidate plane being visible in the optical data; and
determine the angle between the trailer and the tractor based at least on the at least one candidate plane and the one or more markers.

12. The non-transitory computer readable storage medium of claim 11 , wherein the optical sensor comprises at least one of the following: an infrared sensor, a multi spectral camera, and a RADAR.

13. The non-transitory computer readable storage medium of claim 11 , wherein:
the points of the point cloud data are representative of the location of a surface of the trailer in space, and
the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause at least one computing device to:
reduce noise in the points of the point cloud data representative of the location of the surface of the trailer that are inconsistent with an expected shape of the trailer,
wherein determining the at least one candidate plane is performed based on the points having reduced noise.

14. The non-transitory computer readable storage medium of claim 11 , wherein the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause at least one computing device to:
using a plane fitting algorithm to detect the at least one candidate plane in the point cloud data.

15. The non-transitory computer readable storage medium of claim 14 , wherein:
the plane fitting algorithm is further configured to detect a plurality of candidate planes in the point cloud data including the at least one candidate plane.

16. The non-transitory computer readable storage medium of claim 15 , wherein the non-transitory computer readable storage medium further has stored thereon instructions that, when executed, cause at least one computing device to:
for each one of the plurality of candidate planes, determine a distance from a current one of the candidate planes to a first one of the one or more markers, and
determine that the current one of the candidate planes belongs to the trailer in response to the distance from the current one of the candidate planes to the first marker being less than a threshold distance.

17. A method for determining an angle between a tractor and a trailer, comprising:
receiving optical data from an optical sensor, the optical data comprising point cloud data, the optical sensor configured to be mounted on the tractor and generate the optical data indicative of the angle formed between the trailer and the tractor;
detecting one or more markers on a surface of the trailer via identifying points from the point cloud data having a reflectance value lower than a threshold reflectance value;
determining at least one candidate plane representative of the surface of the trailer based on the point cloud data, the candidate plane being visible in the optical data; and
determining the angle between the trailer and the tractor based at least on the at least one candidate plane and the one or more markers.

18. The method of claim 17 , further comprising:
constructing an objective function,
optimizing the point cloud data usi

Assignees

• Tusimple Inc

Inventors

• Luo Yi
• Han Xiaoling
• Mei Xue

Classifications

• G06V20/588 Primary

  Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road · CPC title

• G08G1/167

  Driving aids for lane monitoring, lane changing, e.g. blind spot detection · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T2207/30204

  Marker · CPC title