Systems and methods for a haptically-enabled projected user interface

Patent metadata

Field	Value
Publication number	US-9965034-B2
Application number	US-201314143675-A
Country	US
Kind code	B2
Filing date	Dec 30, 2013
Priority date	Dec 30, 2013
Publication date	May 8, 2018
Grant date	May 8, 2018

Abstract

One illustrative system disclosed herein includes a sensor configured to detect a user input associated with a projected user interface and transmit a sensor signal associated with the user interaction, and a processor in communication with the sensor, the processor configured to: receive the sensor signal, determine a remote haptic effect based at least in part on the user input, generate a haptic signal based at least in part on the remote haptic effect, and transmit the haptic signal. The illustrative system further includes a haptic output device in communication with the processor, the haptic output device configured to receive the haptic signal and output the remote haptic effect.

First claim

What is claimed:

1. A system comprising:
a sensor configured to detect an interaction with a projected user interface that is projected onto a surface and transmit a sensor signal associated with the interaction;
a processor in communication with the sensor;
a memory device on which instructions executable by the processor are stored to cause the processor to:
receive the sensor signal;
determine a characteristic of the surface based at least in part on at least one sensor signal from at least one sensor;
determine a remote haptic effect based at least in part on the interaction and the characteristic of the surface;
generate a haptic signal based at least in part on the remote haptic effect; and
transmit the haptic signal; and

a haptic output device in communication with the processor and remote from the surface, the haptic output device configured to receive the haptic signal and output the remote haptic effect by projecting an emission comprising at least one of a solid, liquid, gas, plasma, or sound pressure wave outwardly from the haptic output device.

2. The system of claim 1 , wherein the sensor comprises one or more of a camera, an accelerometer, a gyroscope, a depth sensor, or a touch-sensitive surface.

3. The system of claim 1 , further comprising a projector for generating the projected user interface.

4. The system of claim 1 , wherein the haptic output device comprises one or more of an ultrasonic transducer, a speaker, or a jet configured to project the emission outwardly from the haptic output device to create the remote haptic effect.

5. The system of claim 1 , wherein the processor and memory device are positioned in a hand-held computing device that comprises a smartphone, tablet, e-reader, or pocket organizer.

6. The system of claim 1 , wherein the projected user interface comprises a hidden interface component, and the interaction is associated with the hidden interface component.

7. The system of claim 1 , wherein the characteristic comprises a material of the surface.

8. The system of claim 1 , wherein the memory device further includes instructions executable by the processor to cause the processor to determine the remote haptic effect based at least in part on a distance between the haptic output device and a body part to which the remote haptic effect is to be applied.

9. The system of claim 1 , wherein the characteristic comprises a fundamental frequency of the surface.

10. The system of claim 1 , wherein the characteristic comprises a topology of the surface.

11. The system of claim 1 , wherein the sensor, the processor, and the haptic output device are disposed within a single device.

12. The system of claim 1 , wherein the emission comprises the sound pressure wave and the haptic output device is configured to project the sound pressure wave outwardly to create the remote haptic effect.

13. The system of claim 1 , wherein the emission comprises the solid or the liquid and the haptic output device is configured to project the solid or the liquid outwardly to create the remote haptic effect.

14. The system of claim 1 , wherein the emission comprises the gas and the haptic output device is configured to project the gas outwardly to create the remote haptic effect.

15. The system of claim 1 , wherein the haptic output device is a first haptic output device, the remote haptic effect is a first haptic effect, and further comprising a second haptic output device coupled to the surface and configured to output a second haptic effect.

16. The system of claim 15 , wherein the memory device further comprises instructions executable by the processor to cause the processor to:
determine the second haptic effect based at least in part on the interaction; and
cause the second haptic output device to output the second haptic effect concurrently with the first haptic output device outputting the first haptic effect.

17. The system of claim 15 , wherein the interaction is a first interaction, and wherein the memory device further comprises instructions executable by the processor to cause the processor to:
determine a haptic effect to output based on a second interaction;
select a particular haptic-output device from among a group of haptic output devices comprising the first haptic output device and the second haptic output device, wherein the particular haptic-output device is selected from among the group of haptic output devices based on the haptic effect; and
cause the particular haptic-output device to output the haptic effect.

18. A method comprising:
receiving, by a processor and from a sensor, a sensor signal associated with an interaction with a projected user interface that is projected onto a surface;
determining, by the processor, a characteristic of the surface based at least in part on at least one sensor signal from at least one sensor;
determining, by the processor, a remote haptic effect based at least in part on the interaction and the characteristic of the surface; and
causing, by the processor, a haptic output device that is positioned remotely from the surface to output the remote haptic effect to a body part by projecting an emission comprising at least one of a solid, liquid, gas, plasma, or sound pressure wave to the body part.

19. The method of claim 18 , wherein the sensor comprises one or more of a camera, an accelerometer, a gyroscope, a depth sensor, or a touch-sensitive surface.

20. The method of claim 18 , wherein the haptic output device comprises one or more of an ultrasonic transducer, a speaker, or a jet configured to project the emission outwardly from the haptic output device toward the body part.

21. A non-transient computer readable medium comprising program code, which when executed by a processor is configured to cause the processor to:
receive a sensor signal associated with an interaction with a projected user interface that is projected onto a surface;
determine a characteristic of the surface based at least in part on at least one sensor signal from at least one sensor;
determine a remote haptic effect based at least in part on the interaction and the characteristic of the surface;
generate a haptic signal based at least in part on the remote haptic effect; and
transmit the haptic signal associated with the remote haptic effect to a haptic output device positioned remotely from the surface, the haptic signal being configured cause the haptic output device to output the remote haptic effect by projecting an emission comprising at least one of a solid, liquid, gas, plasma, or concentrated pressure wave outwardly from the haptic output device.

22. The non-transient computer readable medium of claim 21 , wherein the haptic output device comprises one or more of an ultrasonic transducer, a speaker, or a jet configured to project the emission to create the remote haptic effect.

Assignees

• Immersion Corp

Inventors

• Levesque Vincent
• Modarres Ali
• Weddle Amaya B
• Birnbaum David M

Classifications

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title