Vehicle control data generation method, vehicle controller, vehicle control system, vehicle learning device, vehicle control data generation device, and memory medium

Patent metadata

Field	Value
Publication number	US-11679784-B2
Application number	US-202016950242-A
Country	US
Kind code	B2
Filing date	Nov 17, 2020
Priority date	Jan 9, 2020
Publication date	Jun 20, 2023
Grant date	Jun 20, 2023

Abstract

A vehicle control data generation method is provided. A self-driving mode of a vehicle automatically generates a command value of a propelling force produced by a propelling force generator independently of an accelerator operation. An execution device provides a greater reward based on an obtained state of the vehicle when a standard of a characteristic of the vehicle is met than when the standard is not met. Providing the reward changes a reward that is provided when the characteristic of the vehicle is a predetermined characteristic in the self-driving mode such that the changed reward differs from a reward that is provided when the characteristic of the vehicle is the predetermined characteristic in a manual driving mode.

First claim

What is claimed is:

1. A vehicle control data generation method, wherein
the generation method is applied to a vehicle that allows for a self-driving mode and a manual driving mode, the self-driving mode automatically generating a command value of a propelling force produced by a propelling force generator independently of an accelerator operation, the manual driving mode generating the command value of the propelling force produced by the propelling force generator in accordance with the accelerator operation,
relationship defining data defines a relationship between a state of the vehicle and an action variable that includes a variable related to an operation of the propelling force generator,
the generation method, with the relationship defining data stored in a memory device, comprises:
obtaining, by an execution device, the state of the vehicle that is based on a detection value of a sensor;
operating, by the execution device, the propelling force generator;
providing, by the execution device, based on the obtained state of the vehicle, a reward that is greater when a standard of a characteristic of the vehicle is met than when the standard is not met, wherein the providing the reward includes:
a first process that increases the reward when a standard related to followability for the command value of the propelling force is met than when the standard is not met, and
a second process that increases the reward when a standard related to an absolute value of an acceleration of the vehicle in a front-rear direction is met than when the standard is not met; and

updating, by the execution device, the relationship defining data by inputting to a predetermined update map the obtained state of the vehicle, a value of the action variable used to operate the propelling force generator, and the reward corresponding to the operation of the propelling force generator,

wherein:
the update map outputs the updated relationship defining data so as to increase an expected return for the reward in a case where the propelling force generator is operated in accordance with the relationship defining data, and
the providing the reward includes changing the reward that is provided when the characteristic of the vehicle is a predetermined characteristic in the self-driving mode such that the changed reward differs from the reward that is provided when the characteristic of the vehicle is the predetermined characteristic in the manual driving mode, wherein the changing the reward includes changing, so as to be advantageous in obtaining the increased reward when the absolute value of acceleration is smaller in the self-driving mode as compared with the manual driving mode, at least one of the first process or the second process.

2. The vehicle control data generation method according to claim 1 , further comprising generating, by the execution device, based on the updated relationship defining data, control mapping data by associating the state of the vehicle with the value of the action variable that maximizes the expected return, wherein
the state of the vehicle is input to the control mapping data, which outputs the value of the action variable that maximizes the expected return.

3. A vehicle controller, comprising the memory device and the execution device in the vehicle control data generation method according to claim 1 , wherein
the operating the electronic device includes operating, based on the relationship defining data, the propelling force generator in accordance with the value of the action variable corresponding to the state of the vehicle obtained by obtaining the state of the vehicle.

4. A vehicle control system, comprising the execution device and the memory device in the vehicle controller according to claim 3 , wherein
the execution device includes a first execution device mounted on the vehicle and a second execution device that is an out-of-vehicle device,
the first execution device is configured to execute at least the obtaining the state of the vehicle and the operating the electronic device, and
the second execution device is configured to execute at least the updating the relationship defining data.

5. A vehicle controller, comprising the first execution device in the vehicle control system according to claim 4 .

6. A vehicle learning device, comprising the second execution device in the vehicle control system according to claim 4 .

7. A vehicle control data generation device, comprising a memory device and an execution device, wherein
the generation device is applied to a vehicle that allows for a self-driving mode and a manual driving mode, the self-driving mode automatically generating a command value of a propelling force produced by a propelling force generator independently of an accelerator operation, the manual driving mode generating the command value of the propelling force produced by the propelling force generator in accordance with the accelerator operation,
the memory device is configured to store relationship defining data that defines a relationship between a state of the vehicle and an action that includes a variable related to an operation of the propelling force generator,
the execution device, with the relationship defining data stored in the memory device, is configured to execute:
an obtaining process that obtains the state of the vehicle that is based on a detection value of a sensor;
an operation process that operates the propelling force generator;
a reward calculating process that provides, based on the state of the vehicle obtained by the obtaining process, a reward that is greater when a standard of a characteristic of the vehicle is met than when the standard is not met, wherein the providing the reward includes:
a first process that increases the reward when a standard related to followability for the command value of the propelling force is met than when the standard is not met, and
a second process that increases the reward when a standard related to an absolute value of an acceleration of the vehicle in a front-rear direction is met than when the standard is not met; and

an update process that updates the relationship defining data by inputting to a predetermined update map the state of the vehicle obtained by the obtaining process, a value of the action variable used to operate the propelling force generator, and the reward corresponding to the operation of the propelling force generator,
wherein:
the update map outputs the updated relationship defining data so as to increase an expected return for the reward in a case where the propelling force generator is operated in accordance with the relationship defining data, and
the reward calculating process includes a changing process that changes a reward that is provided when the characteristic of the vehicle is a predetermined characteristic in the self-driving mode such that the changed reward differs from the reward that is provided when the characteristic of the vehicle is the predetermined characteristic in the manual driving mode, wherein the changing the reward includes changing, so as to be advantageous in obtaining the increased reward when the absolute value of acceleration is smaller in the self-driving mode as compared with the manual driving mode, at least one of the first process or the second process.

8. A non-transitory computer readable memory medium that stores a program causing an execution device to execute a vehicle control data generation process, wherein
the generation process is applied to a vehicle that allows for a self-driving mode and a manual driving mode, the self-driving mode automatically generating a command value of a propelling force produced by a propelling force generator independently of an accelerator opera

Assignees

• Toyota Motor Co Ltd

Inventors

• Hashimoto Yosuke
• Katayama Akihiro
• Oshiro Yuta
• Sugie Kazuki
• Oka Naoya

Classifications

• B60W60/0051 Primary

  from occupants to vehicle · CPC title

• B60W2540/10

  Accelerator pedal position · CPC title

• B60W50/0098

  Details of control systems ensuring comfort, safety or stability not otherwise provided for · CPC title

• B60W10/06 Primary

  including control of combustion engines · CPC title

• B60R16/0231 Primary

  Circuits relating to the driving or the functioning of the vehicle (monitoring tyres B60C23/00; indicating overspeed B60K31/00; for dash boards B60K37/00, B60Q3/10; for indicating emergencies B60Q1/52; brake control systems B60T; registering or indicating the working of vehicles G07C5/00; measuring distance G01C, e.g. combinations of speed and distance G01C23/00; engine indicators G01L; measuring speed or acceleration G01P) · CPC title