Modeling a neuronal controller exhibiting human postural sway

Patent metadata

Field	Value
Publication number	US-11540781-B2
Application number	US-202016800646-A
Country	US
Kind code	B2
Filing date	Feb 25, 2020
Priority date	Mar 29, 2019
Publication date	Jan 3, 2023
Grant date	Jan 3, 2023

Abstract

Conventionally, a neuronal controller located inside the central nervous system governing the maintenance of the upright posture of the human body is designed from a control system perspective using proportional-integral-derivative (PID) controllers, wherein human postural sway is modeled either along a sagittal plan or along a frontal plane separately resulting in limited insights on intricacies of a governing neuronal controller. Also, existing neuronal controllers using a reinforcement learning (RL) paradigm are based on complex actor-critic on-policy algorithms. Analyzing human postural sway is critical to detect markers for progression of balance impairments. The present disclosure facilitates modelling the neuronal controller using a simplified RL algorithm, capable of producing postural sway characteristics in both sagittal and frontal plane together. The Q-learning technique of the RL paradigm is employed for learning an optimal state-action value (Q-value) function for a tuneable Markov Decision Process (MDP) model.

First claim

What is claimed is:

1. A processor implemented method for modeling a neuronal controller exhibiting human postural sway, the method comprising the steps of:
modeling, by one or more hardware processors, the neuronal controller in the form of a Reinforcement Learning (RL) agent based on an inverted pendulum with 1 Degree Of Freedom (1 DOF) representing a first mechanical model of a human body in the form of a dynamical system, wherein the RL agent is trained using a Q-learning technique to learn an optimal state-action value (Q-value) function for a tuneable Markov Decision Process (MDP) model, wherein the modeled neuronal controller is configured to reproduce Center Of Pressure (COP) characteristics in the human body either along a sagittal plane or along a frontal plane separately;
deriving, by the one or more hardware processors, dynamical equations of a Spherical Inverted Pendulum (SIP) with respect to a global coordinate system, wherein the SIP represents a second mechanical model of the human body that exhibits postural sway along both the frontal plane and the sagittal plane together, wherein the dynamical equations are derived by using Lagrange's equations with two independent state variables (θx and θy) being angular deviation of the SIP about a pivot joint and along x and y axes respectively of the global coordinate system, wherein the pivot joint characterizes an ankle joint of the human body; and
modeling, by the one or more hardware processors, the human postural sway both along the sagittal plane and along the frontal plane together using the modeled neuronal controller and the derived dynamical equations of the SIP by tuning (i) a reward function comprised in the modeled neuronal controller and (ii) a set of parameters to balance the SIP such that the postural sway characteristics generated by the modeled neuronal controller match the postural sway characteristics of one or more control subjects, wherein the set of parameters include parameters of the MDP model and parameters associated with physiology of the human body.

2. The processor implemented method of claim 1 , wherein the Q-learning technique is configured to learn to generate a torque representing an action for each state of the inverted pendulum by:
receiving proprioceptive inputs in the form of angle and angular velocity of the pivot joint of the inverted pendulum; and
generating the torque at the pivot joint of the inverted pendulum based on the received proprioceptive inputs to maintain an upright posture of the inverted pendulum.

3. The processor implemented method of claim 1 , wherein the step of modeling a neuronal controller comprises reproducing the Center Of Pressure (COP) characteristics in the human body either along the sagittal or along the frontal plane separately in accordance with a dynamical equation based on overall mass of the human body being concentrated at the point of Center Of Mass (COM) located at the 2 nd lumbar vertebrae of the human body, an approximate height of the 2 nd lumbar vertebrae, moment of inertia of the inverted pendulum, angle of the inverted pendulum with respect to the direction of gravity, gravitational acceleration, a stiffness constant and a damping constant, wherein the stiffness constant and the damping constant denote pivot joint properties of the inverted pendulum.

4. The processor implemented method of claim 3 , wherein the step of modeling a neuronal controller comprises reproducing the Center Of Pressure (COP) characteristics in the human body either along the sagittal or along the frontal plane separately in accordance with the dynamical equation represented as:

I
⁢

d
2

⁢
θ

dt
2

=

τ
+

mgL
⁢
sin
⁢
θ

-

B
⁢
θ

-

K
⁢

θ
.

,

wherein I=mL 2 , and wherein
m represents overall mass of the human body being concentrated at the point of Center Of Mass (COM) located at the 2 nd lumbar vertebrae of the human body;
L represents the approximate height of the 2 nd lumbar vertebrae;
θ represents angle of the inverted pendulum with respect to the direction of gravity;
g represents gravitational acceleration;
I represents moment of inertia of the inverted pendulum;
K represents the stiffness constant; and
B represents the damping constant.

5. The processor implemented method of claim 2 , wherein the parameters of the MDP model include:
n θ representing resolution of states in θ domain (from θ max to −θ max );
n {dot over (θ)} representing resolution of states in {dot over (θ)} domain (from {dot over (θ)} max to −{dot over (θ)} max );
n A representing resolution of states in τ domain (from τ max to −τ max );
τ max representing the maximum torque exertable on the pivot joint or the boundary of the τ domain;
θ max representing the boundary of the θ domain;
{dot over (θ)} max representing a finite boundary of the {dot over (θ)} domain;
p α representing property of a curve from which torque values in the τ domain is sampled;
p θ representing property of a curve from which boundaries of states of the θ domain is sampled; and
p {dot over (θ)} representing property of a curve from which boundaries of the states of the {dot over (θ)} domain is sampled.

6. The processor implemented method of claim 2 , wherein the parameters associated with physiology of the human body include:
White Gaussian Noise (WGN) added to the generated torque to represent a real world noisy biological system;
filtering factor (λ) added to portray signaling characteristics of a neuromuscular junction; and
Scaling Factor (SF) introduced to scale down the magnitude of the generated torque for each state of the inverted pendulum after the training of the RL agent is completed, wherein completion of the training is represented by a balanced SIP.

7. The processor implemented method of claim 6 , wherein a relationship between the generated torque by the modeled neuronal controller and the torque applied to the dynamical system based on the parameters associated with physiology of the human body is represented as:

τ
⁡
(
t
)

=

λ
⁢

τ
1

(
t
)

+

(

Assignees

• Tata Consultancy Services Ltd

Inventors

• Sinha Aniruddha
• Mazumder Oishee
• Chatterjee Debatri
• Vaddadi Srinivasa Chakravarthy
• Biswas Dipayan

Classifications

• A61B5/725

  using specific filters therefor, e.g. Kalman or adaptive filters (specific diagnostics methods using using bioelectric or biomagnetic signals A61B5/316) · CPC title

• A61B5/7246

  using correlation, e.g. template matching or determination of similarity · CPC title

• A61B5/7225

  Details of analogue processing, e.g. isolation amplifier, gain or sensitivity adjustment, filtering, baseline or drift compensation (input circuits for detecting, measuring, or recording bioelectric or biomagnetic signals A61B5/30; specific diagnostic methods using bioelectric or biomagnetic signals A61B5/316) · CPC title

• G16H50/50 Primary

  for simulation or modelling of medical disorders · CPC title

• A61B5/7203

  for noise prevention, reduction or removal · CPC title