Autonomous machine control using neural networks

Patent metadata

Field	Value
Publication number	US-12511515-B2
Application number	US-202318133986-A
Country	US
Kind code	B2
Filing date	Apr 12, 2023
Priority date	Apr 28, 2020
Publication date	Dec 30, 2025
Grant date	Dec 30, 2025

Abstract

Apparatuses, systems, and techniques to infer a sequence of actions to perform using one or more neural networks trained, at least in part, by optimizing a probability distribution function using a cost function, wherein the probability distribution represents different sequences of actions that can be performed. In at least one embodiment, a model predictive control problem is formulated as a Bayesian inference task to infer a set of solutions.

First claim

What is claimed is:

1 . A processor, comprising:
one or more circuits to use one or more neural networks to update a control parameter for an autonomous system according to one or more sequences of actions inferenced based, at least in part, on determining a relative probability of the one or more sequences of actions satisfying one or more criteria.

2 . The processor of claim 1 , wherein the one or more sequences of actions are represented as posterior distributions determined using variational inferencing.

3 . The processor of claim 1 , wherein the one or more circuits are to update the control parameter based, at least in part, on:
sampling a set of costs; determining an approximate posterior distribution based at least in part on the set of costs; sampling one or more policy parameters, one or more control inputs, and one or more observed states; and shifting a candidate posterior distribution based at least in part on the approximate posterior distribution.

4 . The processor of claim 3 , wherein shifting the candidate posterior distribution comprises determining a perturbation direction that decreases Kullback-Leibler divergence between the approximate posterior distribution and a target distribution.

5 . The processor of claim 3 , wherein determining the approximate posterior distribution comprises minimizing a Kullback-Leibler divergence.

6 . The processor of claim 1 , wherein the one or more sequences of actions are to be inferred based at least in part on selecting a set of policy parameters based at least in part on their respective weights.

7 . The processor of claim 1 , wherein the one or more sequences of actions are to navigate a vehicle to a destination.

8 . A system, comprising:
one or more processors to use one or more neural networks to update a control parameter for an autonomous system according to, at least in part, relative probabilities of sequences of actions satisfying one or more criteria; and one or more memories to store parameters of the one or more neural networks.

9 . The system of claim 8 , wherein the one or more processors are to determine the sequences of actions by at least performing a Stein variational inference procedure to approximate one or more distribution through density of one or more sets of particles.

10 . The system of claim 8 , wherein the one or more processors are to determine the relative probabilities of the sequences of actions by at least transforming a problem of estimating a posterior distribution into an optimization task.

11 . The system of claim 8 , wherein the one or more processors are to determine the relative probabilities of the sequences of actions by at least:
determining an approximate posterior distribution with a set of particles; and iteratively updating the set of particles to more closely match a target distribution.

12 . The system of claim 11 , wherein iteratively updating the set of particles comprises minimizing a Kullback-Leibler divergence between the approximate posterior distribution and the target distribution.

13 . The system of claim 11 , wherein the relative probabilities of the sequences of actions are based at least in part on weights of the set of particles.

14 . A machine-readable medium having stored thereon a set of instructions, which if performed by one or more processors, cause the one or more processors to at least:
use one or more neural networks to update a control parameter for an autonomous system based, at least in part, on determining a relative probability of one or more sequences of actions satisfying one or more criteria using a Bayesian inferencing formulation.

15 . The machine-readable medium of claim 14 , wherein the relative probability of the one or more sequences of actions are determined based at least in part on updating the relatively probabilities using Stein variational gradient descent.

16 . The machine-readable medium of claim 15 , wherein the Stein variational gradient descent utilizes a kernel to factorize a multi-dimensional input into a sum of kernels.

17 . The machine-readable medium of claim 14 , wherein the one or more sequences of actions are to be inferred based at least in part on selecting a set of policy parameters based at least in part on their respective weights.

18 . The machine-readable medium of claim 17 , wherein the set of policy parameters are selected based at least in part on highest weight.

19 . The machine-readable medium of claim 17 , wherein the set of policy parameters are selected proportional to the respective weights.

20 . A processor, comprising:
one or more circuits to use one or more neural networks to determine one or more sequences of actions represented as posterior distributions determined using variational inferencing over one or more policy parameters, one or more cost functions, and one or more state observations.

21 . The processor of claim 20 , wherein the one or more sequences of actions are determined based, at least in part, on a relative probability of the one or more sequences of actions satisfying one or more criteria.

22 . The processor of claim 20 , wherein the one or more circuits are to determine the one or more sequences of actions by at least:
sampling a set of costs; determining an approximate posterior distribution; sampling one or more policy parameters, one or more control inputs, and one or more observed states; and shifting a candidate posterior distribution based at least in part on the approximate posterior distribution.

23 . The processor of claim 22 , wherein shifting the candidate posterior distribution comprises determining a perturbation direction that decreases Kullback-Leibler divergence between the approximate posterior distribution and a target distribution.

24 . The processor of claim 22 , wherein determining the approximate posterior distribution comprises minimizing a Kullback-Leibler divergence.

25 . The processor of claim 20 , wherein the one or more sequences of actions are to be inferred based at least in part on selecting a set of policy parameters based at least in part on their respective weights.

26 . The processor of claim 20 , wherein the one or more sequences of actions are to navigate a vehicle to a destination.

27 . A system, comprising:
one or more processors to use one or more neural networks to determine one or more sequences of actions represented as posterior distributions determined using variational inferencing over one or more policy parameters, one or more cost functions, and one or more state observations.

28 . The system of claim 27 , wherein the one or more processors are to determine the one or more sequences of actions by at least performing a Stein variational inference procedure to approximate one or more distribution through density of one or more sets of particles.

29 . The system of claim 27 , wherein the one or more processors are to determine the relative probability of the one or more sequences of actions comprises transforming a problem of estimating a posterior distribution into an optimization task.

30 . The system of claim 27 , wherein the one or more processors are to determine the relative probability of the one or more sequences of actions by at least:
determining an approximate posterior distribution with a set

Assignees

• Nvidia Corp

Inventors

• Lambert Alexander Conrad
• Fishman Adam Harper
• Fox Dieter
• Boots Byron
• Tozeto Ramos Fabio

Classifications

• G05D1/00

  Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) · CPC title

• G05D2101/15

  using machine learning, e.g. neural networks · CPC title

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06N5/04

  Inference or reasoning models · CPC title