Method and system for providing roughness haptic sense of virtual object by using space-time encoding

Patent metadata

Field	Value
Publication number	US-12056281-B2
Application number	US-202118017408-A
Country	US
Kind code	B2
Filing date	Mar 17, 2021
Priority date	Jul 22, 2020
Publication date	Aug 6, 2024
Grant date	Aug 6, 2024

Abstract

A method and a system for providing a roughness haptic sense of a virtual object by using space-time encoding are provided. The method for providing a roughness haptic sense of a virtual object may comprise the steps of: displaying virtual objects to a user; tracking the position of the user's hand; predicting a virtual object, which is most likely to collide with the hand, among the virtual objects; and when the hand touches a haptic sense providing device, controlling the haptic sense providing device on the basis of the surface roughness of the predicted virtual object to feed back a roughness haptic sense to the user.

First claim

The invention claimed is:

1. A method of providing a roughness haptic sense of a virtual object, the method comprising:
displaying virtual objects to a user;
tracking a position of a hand of the user;
predicting a virtual object most likely to collide with the hand among the virtual objects; and
in response to the hand contacting a haptic sense providing device, controlling the haptic sense providing device according to a surface roughness of the predicted virtual object to feed back a roughness haptic sense to the user,
wherein the predicting of the virtual object comprises:
predicting a region of interest of the user based on a gaze direction of the user; and
predicting the virtual object comprising the region of interest to be a virtual object most likely to collide with the hand.

2. The method of claim 1 , wherein the feeding back of the roughness haptic sense comprises:
rotating or moving an end effector of the haptic sense providing device in contact with the hand according to the surface roughness of the predicted virtual object and a moving direction of the hand to perform haptic sense rendering for feeding back the roughness haptic sense.

3. The method of claim 2 , wherein, for the end effector,
polygonal or elliptical shaped protrusions in which an X-axis radius is different from a radius of another axis are formed, or a textured surface on which the protrusions are formed is attached, so that different roughness haptic senses are fed back to the user according to an angle in which the end effector is in contact with the hand.

4. The method of claim 1 , wherein the predicting of the virtual object comprises:
predicting a position and time at which the hand collides with the predicted virtual object based on a position and moving velocity of the hand; and
moving a position of the end effector that makes contact with the hand in the haptic sense providing device and feeds back the roughness haptic sense to the user according to the predicted position and time.

5. A system for providing a roughness of a virtual object, the system comprising:
an HMD for displaying virtual objects to a user;
a virtual object roughness providing device for tracking a position of a hand of the user and predicting a virtual object most likely to collide with the hand among the virtual objects; and
a haptic sense providing device for feeding back a roughness haptic sense to the user according to a surface roughness of the predicted virtual object, in response to the hand contacting the haptic sense providing device,
wherein the virtual object roughness providing device predicts a region of interest of the user based on a gaze direction of the user, and predicts a virtual object including the region of interest to be a virtual object most likely to collide with the hand.

6. The system of claim 5 , wherein the virtual object roughness providing device rotates or moves an end effector of the haptic sense providing device in contact with the hand according to the surface roughness of the predicted virtual object and a moving direction of the hand to perform haptic sense rendering for feeding back the roughness haptic sense.

7. The system of claim 6 , wherein, for the end effector,
polygonal or elliptical shaped protrusions in which an X-axis radius is different from a radius of another axis are formed, or a textured surface on which the protrusions are formed is attached, so that different roughness haptic senses are fed back to the user according to an angle in which the end effector is in contact with the hand.

8. The system of claim 5 , wherein the virtual object roughness providing device predicts a position and time at which the hand collides with the predicted virtual object based on a position and moving velocity of the hand, and according to the predicted position and time, moves a position of the end effector that contacts the hand in the haptic sense providing device and feeds back the roughness haptic sense to the user.

9. A virtual object roughness providing device, the device comprising:
a visual rendering unit for displaying virtual objects to a user using an HMD;
a user tracker for tracking a position of a hand of the user;
an object prediction unit for predicting a virtual object most likely to collide with the hand among the virtual objects; and
a haptic sense rendering unit for, in response to the hand contacting a haptic sense providing device, controlling the haptic sense providing device according to a surface roughness of the predicted virtual object to feed back a roughness haptic sense to the user,
wherein the object prediction unit predicts a region of interest of the user based on a gaze direction of the user, and predicts a virtual object including the region of interest to be a virtual object most likely to collide with the hand.

10. The device of claim 9 , wherein the haptic sense rendering unit rotates or moves an end effector of the haptic sense providing device in contact with the hand according to the surface roughness of the predicted virtual object and a moving direction of the hand to perform haptic sense rendering for feeding back the roughness haptic sense.

11. The device of claim 9 , wherein
the object prediction unit predicts a position and time at which the hand collides with the predicted virtual object based on a position and moving velocity of the hand, and
the haptic sense rendering unit moves a position of the end effector that contacts the hand in the haptic sense providing device and feeds back the roughness haptic sense to the user according to the predicted position and time.

Assignees

• Ewha University—Industry Collaboration Found

Inventors

• Kim Young Jun
• Kim Yae Sol
• Oh Uran

Classifications

• G06F3/0416 Primary

  Control or interface arrangements specially adapted for digitisers · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F2203/014

  Force feedback applied to GUI · CPC title

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title