Method and apparatus for controlling traffic light, method and apparatus for navigating unmanned vehicle, and method and apparatus for training model

Patent metadata

Field	Value
Publication number	US-2024071222-A1
Application number	US-202318503538-A
Country	US
Kind code	A1
Filing date	Nov 7, 2023
Priority date	May 6, 2023
Publication date	Feb 29, 2024
Grant date	—

Abstract

A method for controlling a traffic light, a method and apparatus for navigating an unmanned vehicle and a method and apparatus for training a model are provided. An implementation comprises: generating a reinforced traffic light state parameter according to vehicle state representation information of an unmanned vehicle currently contained in a preset area of a target traffic light and a current traffic light state parameter of the target traffic light; and generating a traffic light control action according to the reinforced traffic light state parameter; where the reinforced traffic light state parameter is used to cause an unmanned vehicle navigation end to generate a reinforced vehicle state parameter according to a reinforced traffic light state and a current vehicle state parameter of a target unmanned vehicle, and generate an unmanned vehicle navigation action according to the reinforced vehicle state parameter.

First claim

What is claimed is:

1 . A method for controlling a traffic light, applied to a traffic light control end communicating with an unmanned vehicle navigation end, the method comprising:
generating a reinforced traffic light state parameter according to vehicle state representation information of an unmanned vehicle currently contained in a preset area of a target traffic light and a current traffic light state parameter of the target traffic light; and generating, according to the reinforced traffic light state parameter, a traffic light control action matching the reinforced traffic light state parameter; wherein the reinforced traffic light state parameter is used to cause the unmanned vehicle navigation end to: generate a reinforced vehicle state parameter according to the reinforced traffic light state parameter and a current vehicle state parameter of a target unmanned vehicle, and generate an unmanned vehicle navigation action matching the reinforced vehicle state parameter according to the reinforced vehicle state parameter.

2 . The method according to claim 1 , wherein the vehicle state representation information is generated by the unmanned vehicle navigation end according to a current vehicle state parameter of the unmanned vehicle contained in the preset area and historical vehicle state representation information at a plurality of previous moments.

3 . The method according to claim 1 , wherein the generating a reinforced traffic light state parameter according to vehicle state representation information of an unmanned vehicle currently contained in a preset area of a target traffic light and a current traffic light state parameter of the target traffic light comprises:
stitching the vehicle state representation information and the current traffic light state parameter into hybrid environment information; and inputting the hybrid environment information into a first encoder, to obtain the reinforced traffic light state parameter.

4 . The method according to claim 1 , wherein the generating, according to the reinforced traffic light state parameter, a traffic light control action matching the reinforced traffic light state parameter comprises:
acquiring associated traffic light state aggregation information of associated traffic lights associated with the target traffic light; and generating the traffic light control action matching the reinforced traffic light state parameter, according to the reinforced traffic light state parameter and the associated traffic light state aggregation information.

5 . The method according to claim 4 , wherein the associated traffic light state aggregation information is generated by:
generating an associated traffic light state matrix according to current traffic light state parameters of the associated traffic lights; and generating the associated traffic light state aggregation information, according to the associated traffic light state matrix, a connectivity parameter of the target traffic light, and a weight matrix of the target traffic light.

6 . The method according to claim 5 , wherein the generating the associated traffic light state aggregation information, according to the associated traffic light state matrix, a connectivity parameter of the target traffic light, and a weight matrix of the target traffic light comprises:
generating, through a first graph neural network, the associated traffic light state aggregation information according to the associated traffic light state matrix, the connectivity parameter of the target traffic light, and the weight matrix of the target traffic light.

7 . The method according to claim 1 , wherein the generating a traffic light control action matching the reinforced traffic light state parameter according to the reinforced traffic light state parameter comprises:
inputting the reinforced traffic light state parameter into a first reinforcement learning model, to obtain the traffic light control action matching the reinforced traffic light state parameter.

8 . The method according to claim 1 , wherein, after the generating a reinforced traffic light state parameter according to vehicle state representation information of an unmanned vehicle currently contained in a preset area of a target traffic light and a current traffic light state parameter of the target traffic light, the method further comprises:
inputting the reinforced traffic light state parameter into a pre-trained goal network to obtain a goal vector, wherein the unmanned vehicle navigation end generates, through a second reinforcement learning model, the unmanned vehicle navigation action matching the reinforced vehicle state parameter according to the reinforced vehicle state parameter; and the goal vector is used to cause the unmanned vehicle navigation end to adjust the second reinforcement learning model according to the goal vector.

9 . The method according to claim 1 , wherein the method further comprises navigating the unmanned vehicle, applied to an unmanned vehicle navigation end communicating with a traffic light control end, the method comprising:
generating a reinforced vehicle state parameter according to a current reinforced traffic light state parameter of a target traffic light that is acquired from the traffic light control end and a current vehicle state parameter of a target unmanned vehicle; and generating, according to the reinforced vehicle state parameter, an unmanned vehicle navigation action matching the reinforced vehicle state parameter, wherein the traffic light control end generates the reinforced traffic light state parameter according to the method of claim 1 .

10 . The method according to claim 9 , wherein, before the generating a reinforced vehicle state parameter according to a current reinforced traffic light state parameter of a target traffic light that is acquired from the traffic light control end and a current vehicle state parameter of a target unmanned vehicle, the method further comprises:
generating vehicle state aggregation information, according to a vehicle state parameter of an unmanned vehicle currently contained in a preset area of the target traffic light; and generating the current vehicle state representation information according to the vehicle state aggregation information and historical vehicle state representation information at a plurality of previous moments, wherein the vehicle state representation information is used to cause the traffic light control end to generate the reinforced traffic light state parameter according to the vehicle state representation information of the unmanned vehicle currently contained in the preset area of the target traffic light and a current traffic light state parameter of the target traffic light.

11 . The method according to claim 10 , wherein the generating vehicle state aggregation information according to a vehicle state parameter of an unmanned vehicle currently contained in a preset area of the target traffic light comprises:
generating, through a second graph neural network, the vehicle state aggregation information according to the vehicle state parameter of the unmanned vehicle currently contained in the preset area of the target traffic light.

12 . The method according to claim 10 , wherein the generating the current vehicle state representation information according to the vehicle state aggregation information and historical vehicle state representation information at a plurality of previous moments comprises:
inputting the vehicle state aggregation information and the historical vehicle state representation information at the plurality of previous moments into a recurrent neural ne

Assignees

• Beijing Baidu Netcom Sci & Tech Co Ltd

Inventors

• Sun Qian
• Zhang Le
• Zhou Jingbo
• Xiong Hui
• Zhang Weijia
• Yu Huan
• Mei Yu
• Ling Weicen

Classifications

• G06F16/35

  Clustering; Classification · CPC title

• G06N3/042

  Knowledge-based neural networks; Logical representations of neural networks · CPC title

• G06N3/092

  Reinforcement learning · CPC title

• G06N3/006

  based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO] · CPC title

• G06N3/045

  Combinations of networks · CPC title