System and method for controlling vehicle using neural network

Patent metadata

Field	Value
Publication number	US-9989964-B2
Application number	US-201615342216-A
Country	US
Kind code	B2
Filing date	Nov 3, 2016
Priority date	Nov 3, 2016
Publication date	Jun 5, 2018
Grant date	Jun 5, 2018

Abstract

A method and a system generate a time-series signal indicative of a variation of the environment in vicinity of the vehicle with respect to a motion of the vehicle and submit the time-series signal to the neural network to produce a reference trajectory as a function of time that satisfies time and spatial constraints on a position of the vehicle. The neural network is trained in to transform time-series signals to reference trajectories of the vehicle. The motion trajectory tracking the reference trajectory while satisfying constraints on the motion of the vehicle is determined and the motion of the vehicle is controlled to follow the motion trajectory.

First claim

The invention claimed is:

1. A method for controlling a vehicle, comprising:
generating, using at least one sensor, a time-series signal indicative of a variation of the environment in vicinity of the vehicle with respect to a motion of the vehicle;
selecting from a memory a neural network trained to transform time-series signals to reference trajectories of the vehicle;
submitting the time-series signal to the neural network to produce a reference trajectory as a function of time that satisfies time and spatial constraints on a position of the vehicle;
determining a motion trajectory tracking the reference trajectory while satisfying constraints on the motion of the vehicle; and
controlling the motion of the vehicle to follow the motion trajectory, wherein at least some steps of the method are performed by a processor operatively connected to the memory and the sensor.

2. The method of claim 1 , wherein the time-series signal includes a derivative of measurements of the sensor.

3. The method of claim 2 , wherein the neural network is trained to receive as an input a vector of a predetermined size, the input vector including elements representing a combination of the time-series signal, first information indicative of a current position of the vehicle, second information indicative of a target position of the vehicle, and third information indicative of the spatial constraints on the current position of the vehicle, further comprising:
determining the current position of the vehicle;
retrieving from the memory the target position of the vehicle and information indicative of the spatial constraints on the current position of the vehicle;
filtering the measurements of the sensor to produce the time-series signal, such that the combination of the time-series signal with the first information, the second information, and the third information results in the vector of the predetermined size;
combining of the time-series signal with the first information, the second information, and the third information to produce an input vector of the predetermined size; and
inputting the input vector into the neural network to produce the reference trajectory.

4. The method of claim 3 , wherein the filtering comprises:
merging and synchronizing measurements of different sensors of the vehicle.

5. The method of claim 1 , wherein the memory stores a set of neural networks, each neural network in the set is trained to consider a different driving style for mapping the time-series signals to reference trajectories of the vehicle, further comprising:
determining a driving style for controlling the vehicle; and
selecting from the memory the neural network corresponding to the determined driving style.

6. The method of claim 5 , wherein the driving style is determined based on an input from a user of the vehicle.

7. The method of claim 5 , wherein the vehicle is a semi-autonomous vehicle, further comprising
learning the driving style of a driver of the vehicle while the vehicle is driven in a manual mode.

8. The method of claim 1 , wherein the neural network includes an encoding subnetwork and a decoding subnetwork trained concurrently with the encoding subnetwork,
wherein the encoding subnetwork is trained to transform the time-series signal to information indicative of the constraints on the motion of the vehicle, and
wherein the decoding subnetwork is trained to produce the reference trajectory from the information indicative of the constraints on the motion of the vehicle.

9. The method of claim 8 , wherein the encoding and decoding subnetworks are recurrent neural networks.

10. The method of claim 8 , wherein the encoding and decoding subnetworks are trained using simulation data for different driving patterns.

11. The method of claim 1 , wherein the reference trajectory defines an objective for the motion of the vehicle, and wherein the motion trajectory includes a set of commands for the actuators of the vehicle to move the vehicle according the objective.

12. A system for controlling a vehicle, comprising:
at least one sensor for sensing the environment in vicinity of the vehicle to generate a time-series signal indicative of a variation of the environment with respect to a motion of the vehicle;
a memory storing a neural network trained to transform time-series signals to reference trajectories of the vehicle;
at least one processor configured for submitting the time-series signal to the neural network selected from the memory to produce a reference trajectory as a function of time that satisfies time and spatial constraints on a position of the vehicle and for determining a motion trajectory tracking the reference trajectory while satisfying constraints on the motion of the vehicle; and
a controller for controlling the motion of the vehicle to follow the motion trajectory.

13. The system of claim 12 , wherein the memory stores a set of neural networks, each neural network in the set is trained to consider a different driving style for mapping the time-series signals to reference trajectories of the vehicle, wherein the processor selects from the memory the neural network corresponding to a determined driving, further comprising:
an input interface for accepting a selection of the driving style from a user of the vehicle.

14. The system of claim 12 , wherein the neural network includes an encoding subnetwork and a decoding subnetwork trained concurrently with the encoding subnetwork,
wherein the encoding subnetwork is trained to transform the time-series signal to information indicative of the constraints on the motion of the vehicle, and
wherein the decoding subnetwork is trained to produce the reference trajectory from the information indicative of the constraints on the motion of the vehicle.

15. The system of claim 14 , wherein the encoding and decoding subnetworks are recurrent neural networks.

16. The system of claim 14 , wherein the encoding and decoding subnetworks are trained using simulation data for different driving patterns.

17. The system of claim 12 , wherein the reference trajectory defines an objective for the motion of the vehicle, and wherein the motion trajectory includes a set of commands for actuators of the vehicle to move the vehicle according the objective.

18. A non-transitory computer readable storage medium embodied thereon a program executable by a processor for performing a method, the method comprising:
generating a time-series signal indicative of a variation of the environment in vicinity of the vehicle with respect to a motion of the vehicle;
selecting a neural network trained to transform time-series signals to reference trajectories of the vehicle;
submitting the time-series signal to the neural network to produce a reference trajectory as a function of time that satisfies time and spatial constraints on a position of the vehicle;
determining a motion trajectory tracking the reference trajectory while satisfying constraints on the motion of the vehicle; and
controlling the motion of the vehicle to follow the motion trajectory.

Assignees

• Mitsubishi Electric Res Laboratories Inc

Inventors

• Berntorp Karl
• Liu Ming-Yu
• Weiss Avishai

Classifications

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/24143

  Distances to neighbourhood prototypes, e.g. restricted Coulomb energy networks [RCEN] · CPC title

• G05B13/027

  using neural networks only · CPC title

• G06N3/049

  Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs · CPC title

• G06N3/08

  Learning methods · CPC title