Dueling bandits algorithm for neuromodulation therapy

Patent metadata

Field	Value
Publication number	US-10821285-B2
Application number	US-201816475856-A
Country	US
Kind code	B2
Filing date	Jan 5, 2018
Priority date	Jan 5, 2017
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

A system, method, and apparatus for identifying optimal or near optimal complex stimulation waveforms for a neurostimulator device or neuromodulation device are disclosed. An example method includes using a dueling bandits algorithm with correlation among stimulation arms to select a batch of stimulation arms for sequential application to a patient during a therapy session. Each of the stimulation arms specifies complex stimulation waveform parameter values. Feedback from applying the stimulation arms to the patient is recorded and used to update feedback reward values corresponding to at least some of the stimulation arms using a stimulation arm correlation index. A second batch of stimulations arms is selected based upon the updated feedback reward values and applied to a patient. The method is iteratively repeated over a number of therapy sessions until an optimal or near optimal batch of stimulation arms (defining complex stimulation waveforms) is determined.

First claim

The invention is claimed as follows:

1. A neurostimulator device comprising:
a stimulation assembly connectable to a plurality of electrodes that are configured to apply stimulation waveforms to stimulate a portion of a patient's body;
a memory storing:
a plurality of stimulation arms for a correlated dueling bandits algorithm, each of the stimulation arms comprising (i) parameters for defining a stimulation waveform, and (ii) information indicative of at least one electrode of the plurality of electrodes that are to be active for applying the stimulation waveform,
a correlation index that specifies a correlation among at least some of the plurality of stimulation arms, and
a feedback index that specifies a feedback reward value for at least some of the plurality of stimulation arms; and

a processor communicatively coupled to the memory and electrically coupled to the stimulation assembly, the processor configured to:
operate the correlated dueling bandits algorithm to select a first batch of a specified number of stimulation arms, among the plurality of stimulation arms,
apply sequentially the stimulation waveforms corresponding to the stimulation arms of the first batch to the patient using the stimulation assembly,
record a feedback reward value for each of the applied stimulation arms of the first batch indicative of an effectiveness of the respective stimulation arm for a therapy for the patient,
update the feedback reward value for at least some of the plurality of stimulation arms using the correlation index specifying the correlation among the stimulation arms,
operate the correlated dueling bandits algorithm to select a second batch of the specified number of stimulation arms, among the plurality of stimulation arms, and
apply sequentially the stimulation waveforms corresponding to the stimulation arms of the second batch to the patient using the stimulation assembly.

2. The neurostimulator device of claim 1 , wherein the processor is configured to filter the plurality of stimulation arms based on at least one of a type of therapy or an indication of a placement of the plurality of electrodes in or on the patient.

3. The neurostimulator device of claim 1 , wherein at least a portion of the correlation index and the feedback index are stored with the corresponding stimulation arms.

4. The neurostimulator device of claim 1 , wherein the feedback reward value for each of the at least some of the plurality of stimulation arms is at least one of estimated from feedback reward values of correlated stimulation arms and determined from subjective or objective patient feedback to the respective stimulation arm.

5. The neurostimulator device of claim 1 , further comprising a sensor configured to measure a response of the patient to the applied stimulation waveforms,
wherein the processor is configured to determine the feedback reward value for each of the applied stimulation arms based on the measured response.

6. The neurostimulator device of claim 1 , wherein the effectiveness of the respective stimulation arm is based on the patient's ability to show improved function in at least one of an autonomic function or a cardiovascular function during application of the stimulation waveforms.

7. The neurostimulator device of claim 6 , wherein the neurostimulator device is configured to provide blood pressure regulation by operating the correlated dueling bandits algorithm to determine optimized stimulation waveforms for application over an area of the patient's spinal cord between the 7th and 8th Thoracic vertebrae for 1 millisecond at 5 to 30 Hz using a monophasic waveform, from 0 to 10 milliamps or up to 70 milliamps.

8. The neurostimulator device of claim 1 , wherein the effectiveness of the respective stimulation arm is based on the patient's ability to show improved musculoskeletal function while utilizing adjunctive rehabilitation devices and equipment during application of the stimulation waveforms.

9. The neurostimulator device of claim 1 , wherein the effectiveness of the respective stimulation arm is based on the patient's ability to move a specified muscle group or limb during application of the stimulation waveform that is defined by the stimulation arm.

10. The neurostimulator device of claim 1 , wherein the parameters for defining the stimulation waveform include at least one of a repetition frequency, a pulse width, a system type, a waveform amplitude, an overlapping frequency, and a waveform mode.

11. The neurostimulator device of claim 1 , wherein the portion of the patient's body includes at least one of a spinal cord, a portion of a spinal cord, a brain, a brainstem, a nerve, a portion of a nerve, a cell body, a ganglia, a nerve root, or targeted end organ or gland.

12. A neurostimulator method comprising:
(i) operating a processor according to a correlated dueling bandits algorithm to select a first batch of a specified number of stimulation arms, among a plurality of stimulation arms that are stored in a memory, each of the stimulation arms comprising (a) parameters for defining a stimulation waveform, and (b) information indicative of at least one electrode of a plurality of electrodes that are to be active for applying the stimulation waveform;
(ii) operating the processor to apply sequentially the stimulation waveforms corresponding to the stimulation arms of the first batch to a patient using a stimulation assembly comprising the plurality of electrodes that are configured to apply the stimulation waveforms to stimulate a portion of the patient's body;
(iii) recording, via the processor, a feedback reward value for each of the applied stimulation arms of the first batch indicative of an effectiveness of the respective stimulation arm for a therapy for the patient;
(iv) updating, via the processor, a feedback reward value for at least some of the plurality of stimulation arms using a correlation index specifying a correlation among at least some of the plurality of stimulation arms;
(v) operating the processor according to the correlated dueling bandits algorithm to select a second batch of the specified number of stimulation arms, among the plurality of stimulation arms; and
(vi) operating the processor to apply sequentially the stimulation waveforms corresponding to the stimulation arms of the second batch to the patient using the stimulation assembly.

13. The neurostimulator method of claim 12 , further comprising:
receiving, in the processor, a subjective score for each of the applied stimulation arms; and
determining, via the processor, the feedback reward value as the subjective score.

14. The neurostimulator method of claim 12 , wherein at least one of the stimulation arms of the second batch is more effective for the therapy than the stimulation arms of the first batch.

15. The neurostimulator method of claim 12 , wherein steps (iii) to (vi) are repeated an n-number of times to converge upon an optimal or near optimal stimulation arm.

16. The neurostimulator method of claim 12 , wherein n is between 2 and 100.

17. A neurostimulator device comprising:
a memory storing:
a plurality of stimulation arms for a correlated dueling bandits algorithm, each of the stimulation arms comprising (i) parameters for defining a stimulation waveform, and (ii) information indicative of at least one electrode of a plurality of electrodes that are to be active for applying the stimulation waveform,
a correlation index that specifies a correlation among at least some of the plurality of stimulation arms, and
a feedback index that specifies a feedback

Assignees

• California Inst Of Techn
• Terrafranca Nicholas A

Inventors

• Burdick Joel W
• Sui Yanan
• Yue Yisong
• Terrafranca Nicholas A

Classifications

• A61B5/395

  Details of stimulation, e.g. nerve stimulation to elicit EMG response · CPC title

• A61B5/389

  Electromyography [EMG] · CPC title

• A61B5/316

  Modalities, i.e. specific diagnostic methods · CPC title

• A61B5/24

  Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof · CPC title

• A61N1/36117 Primary

  for treating hypertension · CPC title