Systems and methods for viewport-based augmented reality haptic effects

Patent metadata

Field	Value
Publication number	US-9690370-B2
Application number	US-201414269357-A
Country	US
Kind code	B2
Filing date	May 5, 2014
Priority date	May 5, 2014
Publication date	Jun 27, 2017
Grant date	Jun 27, 2017

Abstract

One illustrative system disclosed herein includes a display configured to receive a display signal and output an image, and an image capture device configured to capture an area image and transmit an image signal. The illustrative system also includes a processor in communication with the image capture device and the display, the processor configured to: receive the image signal; determine a virtual object based in part on the image signal; determine the display signal based in part on the image signal, wherein the display signal includes data associated with the virtual object; determine a haptic effect based at least in part on the virtual object; and transmit a haptic signal associated with the haptic effect. The illustrative system further includes a haptic output device configured to receive the haptic signal and output the haptic effect.

First claim

What is claimed:

1. A system comprising:
a display configured to receive a display signal and output an image;
an image capture device configured to capture an area image and transmit an image signal;
a processor in communication with the image capture device and the display, the processor configured to execute an augmented reality application, the augmented reality application configured to:
receive the image signal;
determine a virtual object based in part on the image signal;
determine the display signal based in part on the image signal, wherein the display signal is configured to cause an augmented reality environment in which the virtual object overlays at least a portion of the area image to be displayed;
determine a haptic effect based at least in part on a movement of the virtual object in the augmented reality environment;
transmit a haptic signal associated with the haptic effect; and

a haptic output device configured to receive the haptic signal and output the haptic effect.

2. The system of claim 1 , wherein the display comprises a touch screen display.

3. The system of claim 1 , wherein the virtual object comprises: an automobile, a bike, a piece of furniture, an insect, a building, a house, an animal, a computing device, a cartoon character, a logo, or a plant.

4. The system of claim 1 , wherein the haptic output device is coupled to an article configured to be worn by a user.

5. The system of claim 4 , wherein the article comprises: a sleeve, a watch, a bracelet, a wristband, an article of clothing, a jacket, glasses, a glove, a ring, a headband, a hat, or a shoe.

6. The system of claim 1 , wherein the haptic effect comprises one or more of: a vibration, a simulated texture, a change in a perceived coefficient of friction, or an emission of one or more of solids, liquids, gasses, plasmas, or concentrated pressure waves.

7. The system of claim 1 , wherein determining the haptic effect further comprises:
determining a user interaction with the virtual object; and
determining the haptic effect based at least in part on the user interaction.

8. The system of claim 1 , wherein a hand-held device comprises the display and the image capture device.

9. The system of claim 8 , wherein the hand-held device comprises a tablet, a mobile phone, an e-reader, a laptop computer, or a portable gaming device.

10. The system of claim 1 , wherein the haptic effect is configured to simulate the movement of the virtual object along a surface in the augmented reality environment.

11. The system of claim 10 , wherein the surface comprises a body part and the movement comprises the virtual object walking along the body part.

12. A method comprising:
receiving an image signal comprising an area image;
determining a virtual object based in part on the image signal;
determining a display signal based in part on the image signal, wherein the display signal is configured to cause an image to be displayed via a display, the image comprising an augmented reality environment that includes the virtual object overlaying at least a portion of the area image;
transmitting the display signal to the display, wherein the display is configured to output the image;
determining a haptic effect based at least in part on a movement of the virtual object in the augmented reality environment;
transmitting a haptic signal associated with the haptic effect to a haptic output device; and
outputting the haptic effect.

13. The method of claim 12 , wherein the display comprises a touch screen display.

14. The method of claim 12 , wherein the virtual object comprises: an automobile, a bike, a piece of furniture, an insect, a building, a house, an animal, a person, a computing device, a cartoon character, a logo, or a plant.

15. The method of claim 12 , wherein the haptic output device is coupled to an article configured to be worn by a user.

16. The method of claim 15 , wherein the article comprises: a sleeve, a watch, a bracelet, a wristband, an article of clothing, a jacket, glasses, a glove, a ring, a headband, a hat, or a shoe.

17. The method of claim 12 , wherein the haptic effect comprises one or more of: a vibration, a simulated texture, a change in a perceived coefficient of friction, or an emission of one or more of solids, liquids, gasses, plasmas, or concentrated pressure waves.

18. The method of claim 12 , wherein determining the haptic effect further comprises:
determining a user interaction with the virtual object; and
determining the haptic effect based at least in part on the user interaction.

19. A non-transient computer readable medium comprising program code, which when executed by a processor is configured to cause the processor to:
receive an image signal comprising an area image;
determine a virtual object based in part on the image signal;
determine a display signal based in part on the image signal, wherein the display signal is configured to cause an image to be displayed via a display, the image comprising an augmented reality environment that includes the virtual object overlaying at least a portion of the area image;
transmit the display signal to the display, wherein the display is configured to output the image;
determine a haptic effect based at least in part on a movement of the virtual object in the augmented reality environment;
transmit a haptic signal associated with the haptic effect to a haptic output device; and
output the haptic effect.

20. The non-transient computer readable medium of claim 19 , wherein the virtual object comprises: an automobile, a bike, a piece of furniture, an insect, a building, a house, an animal, a person, a computing device, a cartoon character, a logo, or a plant.

21. The non-transient computer readable medium of claim 19 , wherein the haptic output device is coupled to an article configured to be worn by a user.

22. The non-transient computer readable medium of claim 19 , wherein determining the haptic effect further comprises:
determining a user interaction with the virtual object; and
determining the haptic effect based at least in part on the user interaction.

Assignees

• Immersion Corp

Inventors

• Levesque Vincent
• Grant Danny

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title