Methods and systems for scheduling mmWave communications using reinforcement learning

Patent metadata

Field	Value
Publication number	US-12439431-B2
Application number	US-202117354045-A
Country	US
Kind code	B2
Filing date	Jun 22, 2021
Priority date	Jun 22, 2021
Publication date	Oct 7, 2025
Grant date	Oct 7, 2025

Abstract

A controller for scheduling mmWave communication is provided. The controller is programmed to identify a plurality of states, each of the plurality of states indicating status of mmWave communication links among a plurality of nodes, calculate an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges, and select one of the plurality of states based on the converged values for the plurality of states.

First claim

What is claimed is:

1. A controller which is configured to:
receive an intent to communicate from one or more of a plurality of nodes;
identify a plurality of states based at least in part on the one or more intents to communicate, each of the plurality of states indicating status of mmWave communication links among the plurality of nodes;
calculate an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges; and
select one of the plurality of states based on the converged values for the plurality of states, wherein:
the updated value for each of the plurality of states is calculated at least based on a reward value and a transition probability from a first state to a second state; and
the reward value is calculated based on a weight value related to a link to be added or removed and a number of conflicts due to an addition or a removal of the link.

2. The controller of claim 1 , wherein the controller is configured to:
broadcast information about the selected state over a V2X channel.

3. The controller of claim 1 , wherein the updated value for each of the plurality of states is calculated using Bellman equation.

4. The controller of claim 1 , wherein:
each of the mm Wave communication links is associated with a weight parameter; and
the updated value for each of the plurality of states is calculated further based on the weight parameter.

5. The controller of claim 1 , wherein the controller is configured to:
broadcast information about the selected state over a 5.9 GHz V2X channel.

6. The controller of claim 1 , wherein the plurality of nodes include a plurality of connected vehicles.

7. The controller of claim 1 , wherein the plurality of nodes include an edge server.

8. The controller of claim 1 , wherein one or more of the plurality of states include mm Wave communication links that conflict each other.

9. A method comprising:
receiving an intent to communicate from one or more of a plurality of nodes;
identifying a plurality of states based at least in part on the one or more intents to communicate, each of the plurality of states indicating status of mm Wave communication links among the plurality of nodes;
calculating an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges; and
selecting one of the plurality of states based on the converged values for the plurality of states, wherein:
the updated value for each of the plurality of states is calculated at least based on a reward value and a transition probability from a first state to a second state; and
the reward value is calculated based on a weight value related to a link to be added or removed and a number of conflicts due to an addition or a removal of the link.

10. The method of claim 9 , further comprising:
broadcasting information about the selected state over a V2X channel.

11. The method of claim 9 , wherein the updated value for each of the plurality of states is calculated using Bellman equation.

12. The method of claim 9 , wherein:
each of the mm Wave communication links is associated with a weight parameter; and
the updated value for each of the plurality of states is calculated further based on the weight parameter.

13. A vehicle system comprising:
a controller which is configured to:
receive an intent to communicate from one or more of a plurality of nodes;
identify a plurality of states based at least in part on the one or more intents to communicate, each of the plurality of states indicating status of mmWave communication links among the plurality of nodes;
calculate an updated value for each of the plurality of states iteratively based on a value iteration algorithm and a previous value for each of the plurality of states until the updated value for each of the plurality of states converges;
select one of the plurality of states based on the converged values for the plurality of states; and
broadcast the selected state over a V2X channel, wherein:
the updated value for each of the plurality of states is calculated at least based on a reward value and a transition probability from a first state to a second state; and
the reward value is calculated based on a weight value related to a link to be added or removed and a number of conflicts due to an addition or a removal of the link.

Assignees

• Toyota Eng & Mfg North America

Inventors

• Lu Hongsheng
• He Chenyuan
• Cheng Bin
• Shimizu Takayuki

Classifications

• H04W4/40

  for vehicles, e.g. vehicle-to-pedestrians [V2P] · CPC title

• G05B13/0265

  the criterion being a learning criterion · CPC title

• H04W76/14

  Direct-mode setup · CPC title

• H04W76/27

  Transitions between radio resource control [RRC] states · CPC title

• H04W4/46

  for vehicle-to-vehicle communication [V2V] · CPC title