Trailer angle detection using end-to-end learning

Patent metadata

Field	Value
Publication number	US-11077795-B2
Application number	US-201816199851-A
Country	US
Kind code	B2
Filing date	Nov 26, 2018
Priority date	Nov 26, 2018
Publication date	Aug 3, 2021
Grant date	Aug 3, 2021

Abstract

A trailer angle identification system comprises an imaging device configured to capture an image. An angle sensor is configured to measure a first angle of the trailer relative to a vehicle. A controller is configured to process the image in a neural network and estimate a second angle of the trailer relative to the vehicle based on the image. The controller is further configured to train the neural network based on a difference between the first angle and the second angle.

First claim

The invention claimed is:

1. A trailer angle identification system comprising:
an imaging device configured to capture a plurality of images;
an angle sensor configured to measure a first angle of the trailer relative to a vehicle; and
a controller configured to:
process the plurality of images in a neural network;
estimate a second angle of the trailer relative to the vehicle based on the plurality of images;
identify an error in the second angle by comparing the first angle to the second angle for the plurality of images;
train the neural network based on the first angle and the second angle for the plurality of images; and
identify additional image data for training the neutral network that depicts a condition of the image data corresponding to the error.

2. The system according to claim 1 , wherein the controller is further configured to:
label the image with the first angle as an input to the neural network.

3. The system according to claim 1 , wherein the controller is further configured to:
train the neural network to identify an actual angle between the vehicle and the trailer based on the image without the first angle from the angle sensor.

4. The system according to claim 1 , wherein the training comprises:
identifying an error between the first angle and the second angle.

5. The system according to claim 1 , wherein the imaging device is configured to capture the image in a field of view directed at a connection interface of the trailer to the vehicle.

6. The system according to claim 1 , wherein the controller is further configured to:
compare the second angle estimated in the plurality of images to the first angle measured by the angle sensor.

7. The system according to claim 1 , wherein the condition comprises at least one of the trailer angle, an environmental condition, and a lighting condition.

8. The system according to claim 1 , wherein the controller is further configured to:
train the neural network with the additional image data that depicts the condition corresponding to the error condition.

9. A method identifying a trailer angle comprising:
capturing a plurality of images in a field of view;
detecting a first angle with an angle sensor in connection with a vehicle or a trailer;
processing the images in a neural network;
estimating a second angle of an interface between the vehicle and the trailer based on each of the images; and
training the neural network based on the first angle and the second angle for each of the images, wherein the training comprises:
identifying an error between the first angle and the second angle for each of the images;
identifying at least one of the trailer angle, an environmental condition, and a lighting condition associated with the error for each of the images, and
capturing additional images based on at least one of the trailer angle, the environmental condition, and the lighting condition associated with the error.

10. The method according to claim 9 , wherein the angle sensor is configured to communicate an electronic signal to a controller based on the trailer angle formed by the interface.

11. The method according to claim 10 , wherein the field of view is directed at an interface of the trailer to the vehicle.

12. The method according to claim 10 , further comprising:
cropping the images based on a location of the interface in the field of view.

13. The method according to claim 9 , wherein the training further comprises:
processing the additional images with the neural network thereby improving the estimation of the second angle by updating the parameters of the neural network.

14. The method according to claim 13 , wherein the training further comprises:
training the neural network to accurately estimate the trailer angle based on the images without the first angle from the angle sensor.

15. The method according to claim 9 , wherein the additional images depict at least one image condition depicted as the trailer angle, the environmental condition, or the lighting condition associated with the error.

16. A trailer angle identification system comprising:
an imaging device configured to capture image data;
an angle sensor configured to measure a first angle of the trailer relative to a vehicle; and
a controller configured to:
crop the image data generating a cropped image data based on a location of an interface between the vehicle and the trailer in the image data;
process the cropped image data in a neural network;
estimate a second angle of the trailer relative to the vehicle based on the cropped image data; and
train the neural network based the first angle and the second angle;
identify an error in the second angle by comparing the first angle to the second angle for the plurality of images, and
identify additional image data for training the neutral network that depicts a condition of the image data corresponding to the error.

17. The system according to claim 16 , wherein the imaging device is configured to capture the image in a field of view directed at the interface of the trailer to the vehicle.

18. The system according to claim 16 , wherein the controller is further configured to:
train the neural network with the additional image data that depicts the condition corresponding to the error.

Assignees

• Ford Global Tech Llc

Inventors

• Jales Costa Bruno Sielly
• Nariyambut Murali Vidya
• Nooshabadi Saeid
• Nagasamy Vijay

Classifications

• G06N3/045

  Combinations of networks · CPC title

• G06V20/56

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G01B21/22

  for measuring angles or tapers; for testing the alignment of axes · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T3/4046

  using neural networks · CPC title