Systems and methods for haptically-enabled neural interfaces

Patent metadata

Field	Value
Publication number	US-10031580-B2
Application number	US-201614994720-A
Country	US
Kind code	B2
Filing date	Jan 13, 2016
Priority date	Jan 13, 2016
Publication date	Jul 24, 2018
Grant date	Jul 24, 2018

Abstract

One illustrative system disclosed herein includes a processor configured to receive a sensor signal from a neural interface configured to detect an electrical signal associated with a nervous system. The processor is also configured to determine an interaction in with a virtual object in a virtual environment based on the sensor signal. The processor is also configured to determine a haptic effect based at least in part on the interaction with the virtual object in the virtual environment. The processor is also configured to transmit a haptic signal associated with the haptic effect. The illustrative system further includes a haptic output device configured to receive the haptic signal and output the haptic effect.

First claim

What is claimed:

1. A computing device comprising:
a processor; and
a memory comprising program code that is executable by the processor to cause the processor to:
receive a sensor signal from a neural interface configured to detect an electrical signal associated with a nervous system;
determine an interaction with an object based on the sensor signal;
determine a haptic effect based at least in part on the interaction with the object;
transmit a haptic signal to a first device, the first device being configured to receive the haptic signal and output the haptic effect via a first haptic output device; and
transmit the haptic signal to a second device that is remote from the first device, the second device being configured to receive the haptic signal and output the haptic effect via a second haptic output device.

2. The computing device of claim 1 , wherein the neural interface comprises a sensor configured to detect the electrical signal associated with the nervous system and transmit the sensor signal, the sensor comprising at least one of: an electroencephalogram (EEG) electrode, a spectroscopic sensor, a magnetic resonance imaging system, or a capacitive sensor.

3. The computing device of claim 1 , wherein the object is a virtual object in an augmented reality environment, and wherein the memory further comprises program code that is executable by the processor to cause the processor to determine the haptic effect based at least in part on an event in the augmented reality environment.

4. The computing device of claim 1 , wherein the the object is an intermediary device, and the interaction with the object comprises a movement of the intermediary device.

5. The computing device of claim 4 , wherein the haptic effect is configured to simulate a characteristic of a surface contacted by the intermediary device.

6. The computing device of claim 1 , wherein the neural interface is a first neural interface, and wherein the second device comprises a second neural interface usable by a remote user.

7. The computing device of claim 1 , wherein the computing device is the first device.

8. A method comprising:
receiving, by a processor of a computing device, a sensor signal from a neural interface configured to detect an electrical signal associated with a nervous system;
determining, by the processor, an interaction with an object based on the sensor signal;
determining, by the processor, a haptic effect based at least in part on the interaction with the object;
transmitting, by the processor, a haptic signal associated with the haptic effect to a first device, the first device being configured to receive the haptic signal and output the haptic effect via a first haptic output device; and
transmitting, by the processor, the haptic signal associated with the haptic effect to a second device that is remote from the first device, the second device being configured to receive the haptic signal and output the haptic effect via a second haptic output device.

9. The method of claim 8 , wherein the neural interface comprises a sensor configured to detect the electrical signal associated with the nervous system and transmit the sensor signal, the sensor comprising at least one of: an electroencephalogram (EEG) electrode, a spectroscopic sensor, a magnetic resonance imaging system, or a capacitive sensor.

10. The method of claim 8 , wherein the object is a virtual object in an augmented reality environment, and wherein the haptic effect is based at least in part on an event in the augmented reality environment.

11. The method of claim 8 , wherein the object is an intermediary device, and the interaction with the object comprises a movement of the intermediary device.

12. The method of claim 11 , wherein the haptic effect is configured to simulate a characteristic of a surface contacted by the intermediary device.

13. The method of claim 8 , wherein the neural interface is a first neural interface, and wherein the second device comprises a second neural interface.

14. The method of claim 8 , wherein the neural interface is the first haptic output device, and wherein the neural interface outputs the haptic effect by electrically stimulating a portion of the nervous system.

15. The method of claim 8 , further comprising:
determining a stimulus that a user is to perceive based on the interaction with the object, the stimulus being a taste, smell, or sound; and
electrically stimulating a portion of the nervous system using the neural interface to cause the user to perceive the stimulus.

16. The method of claim 8 , wherein the haptic effect is a first haptic effect, and further comprising:
determining a stimulus that a user previously perceived based on the sensor signal from the neural interface;
determining a second haptic effect based on the stimulus that the user previously perceived; and
outputting the second haptic effect.

17. The method of claim 16 , wherein the stimulus that the user previously perceived includes at least one of a smell, a taste, or a sound.

18. The method of claim 8 , further comprising:
determining a stimulus that a user previously perceived based on the sensor signal from the neural interface; and
transmitting information associated with the stimulus to the second device.

19. The method of claim 8 , further comprising:
determining a medical status or physiological status of a user based on the sensor signal from the neural interface; and
transmitting information associated with the medical status or physiological status to the second device.

20. The method of claim 8 , further comprising:
determining a texture of the object; and
configuring the haptic effect to enhance a user's perception of the texture of the object.

21. A non-transitory computer readable medium comprising program code, which when executed by a processor is configured to cause the processor to:
receive a sensor signal from a neural interface configured to detect an electrical signal associated with a nervous system;
determine an interaction with an object based on the sensor signal;
determine a haptic effect based at least in part on the interaction with the object;
transmit a haptic signal associated with the haptic effect to a first device, the first device being configured to receive the haptic signal and output the haptic effect via a first haptic output device; and
transmit the haptic signal associated with the haptic effect to a second device that is remote from the first device, the second device being configured to receive the haptic signal and output the haptic effect via a second haptic output device.

22. The non-transitory computer readable medium of claim 21 , wherein the neural interface comprises the first haptic output device.

23. The non-transitory computer readable medium of claim 21 , wherein the object is a virtual object in an augmented reality environment, and wherein the haptic effect is based at least in part on an event in the augmented reality environment.

24. The non-transitory computer readable medium of claim 21 , wherein the object is a prosthetic limb, and the interaction with the object comprises a movement of the prosthetic limb.

Assignees

• Immersion Corp

Inventors

• Cruz-Hernandez Juan Manuel

Classifications

• G06F3/015 Primary

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G06F2203/011

  Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns · CPC title