Dynamic gloves to convey sense of touch and movement for virtual objects in HMD rendered environments

Patent metadata

Field	Value
Publication number	US-10296086-B2
Application number	US-201514664740-A
Country	US
Kind code	B2
Filing date	Mar 20, 2015
Priority date	Mar 20, 2015
Publication date	May 21, 2019
Grant date	May 21, 2019

Abstract

A system and method of using a peripheral device for interfacing with a virtual reality scene generated by a computer for presentation on a head mounted display. The peripheral device includes a haptic device capable of being placed in contact with a user and a haptic feedback controller for processing instructions for outputting a haptic signal to the haptic device. The haptic feedback controller receiving the instructions from the computer so that haptic feedback of the haptic device changes to correspond to a user's virtual interactions with a virtual object in the virtual reality scene as presented on the head mounted display.

First claim

What is claimed is:

1. A system for processing content for a head mounted display (HMD), comprising,
a user interface device having a haptic device for imparting a haptic feedback to a user, the user interface device configured to receive haptic profiles of virtual objects; and
a computing device interfaced with the user interface device and the HMD, the computing device including,
an input device interface for receiving input from the user interface device or the HMD;
a virtual reality space generation module for rendering a virtual reality scene for the HMD;
an object configuration matrix rendering module for identifying a virtual object in the virtual reality scene, the object configuration matrix rendering module accessing a haptic profile for the virtual object; and
a haptic response processing module for identifying interactions between a virtual representation associated with the user interface device and the virtual object and sending the haptic feedback to the user interface device;

wherein the user interface device is operable to use the haptic profile of the virtual object for imparting the haptic feedback to the user, the haptic feedback drives activation of multiple points of pressure that vary in magnitude across a portion of a finger of the user that contacts the user interface device, the user interface device is configured to maintain the multiple points of pressure with continuous pressure for a duration depending upon said interactions between the virtual representation and the virtual object.

2. The system of claim 1 , wherein the user interface is a glove device that includes the haptic device, the glove envelopes said finger of the user that contacts the user interface device, and the haptic response processing module uses the haptic profile to impart changes to the haptic feedback based on a type of interaction detected.

3. The system of claim 2 , further comprising,
a camera for detecting a location of the glove device in real space and a location of the HMD,
wherein the location of the glove device relative to the HMD in real space identifies a location of the glove device relative to the virtual object in the virtual reality scene;
wherein the relative location of the glove device and the HMD is used to identify interactions between the glove device and the virtual object and define a magnitude or type of the haptic feedback provided to at least one portion of the glove device.

4. The system of claim 2 , further comprising,
at least one inertial sensor disposed on the glove device, the inertial sensor generating motion data usable by the computer to determine a position of the glove device or changes in position of the glove device.

5. The system of claim 1 , wherein the content selected for display in the HMD is accessed from one or both of a local storage of the computing device or over a network.

6. The system of claim 1 , wherein each of a plurality of virtual objects have a respective haptic profile, and each haptic profile defines attributes of the virtual object, and attributes of the virtual object can include one or more of a response to being held, a response to being moved, a response to being crushed, a response to being tossed, a response to being dropped, a response to being felt, a response to being rubbed, a response to being squeezed, a response to being pressed, a response to being smashed, a response to being bumped, a definition of size, a definition of temperature, a definition of weight, a definition associated with audio feedback, a definition associated with pressure or texture, a vibration definition, a surface smoothness, a surface roughness, a definition of resistance, of gradual pressure, or combinations of two or more thereof.

7. The system of claim 1 , wherein the virtual reality space generation module is configured to continually generate additional content for the HMD depending at least on a location that the HMD is moved toward while rendering the virtual reality scene, the additional content further including one or more additional virtual objects;
wherein each additional virtual object includes a corresponding haptic profile.

8. The system of claim 1 , wherein a pressure gradient defines how to drive activation of the multiple points of pressure across the finger of the user such that the multiple points of pressure follow a nonlinear curve of pressure magnitude.

9. The system of claim 1 , wherein the haptic profile defines the haptic feedback following a linear or nonlinear curve based on characteristics of the virtual object which drives activation of the multiple points of pressure.

10. A peripheral device for use in interfacing with a virtual reality scene generated by a computer for presentation on a head mounted display, comprising,
a haptic device capable of imparting haptic feedback to a user, the haptic feedback includes a pressure gradient applied across a portion of a finger of the user that contacts the peripheral device and
a haptic feedback controller for processing instructions for outputting a haptic signal to the haptic device, the haptic feedback controller using a haptic profile of a virtual object for outputting the haptic signal, the haptic feedback controller configured to receive the instructions from the computer and to receive the haptic profile of the virtual object from the computer so that the haptic feedback of the haptic device drives activation of multiple points of pressure that vary in magnitude for the pressure gradient that is applied across the portion of the finger of the user in response to an interaction by a virtual representation of the peripheral device with the virtual object in the virtual reality scene when presented on the head mounted display, the haptic feedback controller is configured to maintain the pressure gradient with continuous pressure for a duration depending upon the interaction by the virtual representation with the virtual object in the virtual reality scene;
wherein the haptic feedback of the haptic device is modified by the haptic feedback controller in response to an additional interaction with the virtual object using the peripheral device.

11. The peripheral device of claim 10 , further comprising a motion sensor coupled to the haptic feedback controller, the motion sensor being capable of detecting a motion of the peripheral device relative to the virtual object and wherein the haptic feedback controller is capable of further modifying the haptic feedback to correspond with the detected motion.

12. The peripheral device of claim 10 , wherein the haptic profile includes at least one of a characterization of a shape of the virtual object, or a characterization of a texture of the virtual object, or a characterization of a motion of the virtual object, or a characterization of size of the virtual object, or a characterization of physics of the virtual object, or a characterization of sound of the virtual object, or a characterization of a vibration of the virtual object, or a characterization of an amount of pressure virtually exerted on the user by the virtual object, or a characterization of a proximity of the virtual object to a second virtual object, or combinations of two or more thereof.

13. The peripheral device of claim 10 , wherein the haptic device includes at least one of:
a compressed gas;
a liquid;
an electric motor;
a linear motor;
an electromagnet;
a piezo-electric device;
a heating element;
a cooling element;
one or more electroactive polymers; or
one or more phase changing materials.

14. The peripheral device of claim 10 , wherein the haptic device includes a fluid.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Rimon Noam
• Mao Xiaodong (Crusoe)
• Ye Xiaoyong

Classifications

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• A63F13/355

  Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title