Dynamic haptic retargeting

Patent metadata

Field	Value
Publication number	US-10290153-B2
Application number	US-201715703093-A
Country	US
Kind code	B2
Filing date	Sep 13, 2017
Priority date	Apr 21, 2016
Publication date	May 14, 2019
Grant date	May 14, 2019

Abstract

Dynamic haptic retargeting can be implemented using world warping techniques and body warping techniques. World warping is applied to improve an alignment between a virtual object and a physical object, while body warping is applied to redirect a user's motion to increase a likelihood that a physical hand will reach the physical object at the same time a virtual representation of the hand reaches the virtual object. Threshold values and/or a combination of world warping a body warping can be used to mitigate negative impacts that may be caused by using either technique excessively or independently.

First claim

What is claimed is:

1. A method comprising:
determining, within a physical environment, a physical location of a physical object;
determining, within a virtual environment mapped to the physical environment, a virtual location of a virtual object;
determining that a physical hand of a user is reaching toward the virtual object;
rendering, within the virtual environment, a virtual hand that represents at least a portion of the physical hand of the user while the physical hand of the user is reaching toward the virtual object;
dynamically adjusting the virtual hand to cause the user to physically reach for the physical object while it appears that the virtual hand is reaching for the virtual object, wherein dynamically adjusting the virtual hand includes dynamically applying a body warping to adjust a location of the virtual hand within the virtual environment, wherein applying the body warping includes:
calculating a warping ratio based on the physical location of the physical object, an initial location of the physical hand of the user, and a current location of the physical hand of the user; and
adjusting the location of the virtual hand within the virtual environment based, at least in part, on the warping ratio.

2. The method of claim 1 , further comprising:
mapping the virtual environment to the physical environment to establish an alignment between the virtual environment and the physical environment.

3. The method of claim 1 , further comprising:
dynamically adjusting the alignment between the virtual environment and the physical environment to reduce the difference between the physical location and the virtual location.

4. The method of claim 1 , wherein dynamically adjusting the virtual hand to cause the user to physically reach for the physical object is based, at least in part, on a difference between the physical location of the physical object and the virtual location of the virtual object.

5. The method of claim 1 , wherein dynamically applying the body warping to adjust the location of the virtual hand within the virtual environment further comprises:
determining a virtual vector that represents a distance and direction between the location of the virtual hand and the virtual location of the virtual object;
determining a physical vector that represents a distance and direction between the current location of the physical hand of the user and the physical location of the physical object; and
dynamically applying the body warping to adjust the location of the virtual hand within the virtual environment based at least in part on a difference between the virtual vector and the physical vector.

6. The method of claim 1 , wherein applying the body warping comprises further comprises:
applying a translation to the virtual hand to adjust the location of the virtual hand within the virtual environment; and
applying a rotation to the virtual hand.

7. The method of claim 1 , further comprising:
repeatedly applying a body warping as the physical hand of the user reaches toward the virtual object such that the location of the virtual hand intersects with the virtual location of the virtual object at substantially the same time that a current location of the physical hand of the user intersects with the physical location of the physical object.

8. A non-transitory machine-readable medium storing instructions which, when executed by one or more machines, cause the one or more machines to perform operations comprising:
determining, within a physical environment, a physical location of a physical object;
determining, within a virtual environment mapped to the physical environment, a virtual location of a virtual object;
determining that a physical hand of a user is reaching toward the virtual object;
rendering, within the virtual environment, a virtual hand that represents at least a portion of the physical hand of the user while the physical hand of the user is reaching toward the virtual object;
dynamically adjusting the virtual hand to cause the user to physically reach for the physical object while it appears that the virtual hand is reaching for the virtual object, wherein dynamically adjusting the virtual hand includes dynamically applying a body warping to adjust a location of the virtual hand within the virtual environment, wherein applying the body warping includes:
calculating a warping ratio based on the physical location of the physical object, an initial location of the physical hand of the user, and a current location of the physical hand of the user; and
adjusting the location of the virtual hand within the virtual environment based, at least in part, on the warping ratio.

9. The machine-readable medium of claim 8 , the operations further comprising:
mapping the virtual environment to the physical environment to establish an alignment between the virtual environment and the physical environment.

10. The machine-readable medium of claim 8 , the operations further comprising:
dynamically adjusting the alignment between the virtual environment and the physical environment to reduce the difference between the physical location and the virtual location.

11. The machine-readable medium of claim 8 , wherein dynamically adjusting the virtual hand to cause the user to physically reach for the physical object is based, at least in part, on a difference between the physical location of the physical object and the virtual location of the virtual object.

12. The machine-readable medium of claim 8 , wherein dynamically applying the body warping to adjust the location of the virtual hand within the virtual environment further comprises:
determining a virtual vector that represents a distance and direction between the location of the virtual hand and the virtual location of the virtual object;
determining a physical vector that represents a distance and direction between the current location of the physical hand of the user and the physical location of the physical object; and
dynamically applying the body warping to adjust the location of the virtual hand within the virtual environment based at least in part on a difference between the virtual vector and the physical vector.

13. The machine-readable medium of claim 8 , wherein applying the body warping comprises further comprises:
applying a translation to the virtual hand to adjust the location of the virtual hand within the virtual environment; and
applying a rotation to the virtual hand.

14. The machine-readable medium of claim 8 , the operations further comprising:
repeatedly applying a body warping as the physical hand of the user reaches toward the virtual object such that the location of the virtual hand intersects with the virtual location of the virtual object at substantially the same time that a current location of the physical hand of the user intersects with the physical location of the physical object.

15. A system comprising:
one or more processors; and
one or more memories storing instructions which, when executed by the one or more hardware processors, cause the one or more hardware processors to perform operations comprising:
determining, within a physical environment, a physical location of a physical object;
determining, within a virtual environment mapped to the physical environment, a virtual location of a virtual object;
determining that a physical hand of a user is reaching toward the virtual object;
rendering, within the virtual environment, a virtual hand that represents at least a portion of the physical hand of the user while the physical hand of the user is reaching toward the virtual object;

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Benko Hrvoje
• Wilson Andrew D
• Ofek Eyal
• Azmandian Mahdi
• Hancock Mark

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title