Real-world object interface for virtual, augmented, and mixed reality (xR) applications

Patent metadata

Field	Value
Publication number	US-10937243-B2
Application number	US-201816116688-A
Country	US
Kind code	B2
Filing date	Aug 29, 2018
Priority date	Aug 29, 2018
Publication date	Mar 2, 2021
Grant date	Mar 2, 2021

Abstract

Systems and methods for providing a real-world object interface in virtual, augmented, and mixed reality (xR) applications. In some embodiments, an Information Handling System (IHS) may include one or more processors and a memory coupled to the one or more processors, the memory including program instructions stored thereon that, upon execution by the one or more processors, cause the IHS to: receive a video frame during execution of an xR application; instruct a user wearing a Head-Mounted Display (HMD) to perform a manipulation of a real-world object detected in the video frame; receive additional video frames; determine whether the user has performed the manipulation by tracking the object in the additional video frames; and execute an operation in response to the determination.

First claim

The invention claimed is:

1. An Information Handling System (IHS), comprising:
one or more processors; and
a memory coupled to the one or more processors, the memory including program instructions stored thereon that, upon execution by the one or more processors, cause the IHS to:
receive a video frame during execution of a virtual, augmented, or mixed reality (xR) application;
instruct a user wearing a Head-Mounted Display (HMD) to perform a manipulation of a specified speed of a real-world object detected in the video frame;
receive additional video frames;
identify whether the user has performed the manipulation by tracking the object in the additional video frames;
determine that the manipulation has been performed with a level of complexity configured to avoid false positives; and
at least one of:
(a) grant access to secured content in response to the user having performed the manipulation at the specified speed; or
(b) deny access to the secured content in response to the user having performed the manipulation at a different speed.

2. The IHS of claim 1 , wherein the additional video frames are captured by an inside-out camera mounted on the HMD.

3. The IHS of claim 1 , wherein the manipulation comprises rotation of the real-world object by a selected angle or in a selected direction.

4. The IHS of claim 3 , wherein the angle or direction are set by the user during a training procedure.

5. The IHS of claim 1 , wherein the manipulation comprises translation of the real-world object along a selected trajectory.

6. The IHS of claim 5 , wherein the trajectory is set by the user during a training procedure.

7. The IHS of claim 1 , wherein the speed is set by the user during a training procedure.

8. The IHS of claim 1 , wherein the real-world object is a coffee mug.

9. The IHS of claim 1 , wherein to instruct the user to perform the manipulation, the program instructions, upon execution, cause the IHS to:
render an image for display by the HMD, wherein the image identifies the real-world object.

10. The IHS of claim 9 , wherein the image indicates a rotation angle for the real-world object.

11. The IHS of claim 9 , wherein the image indicates a trajectory for the real-world object.

12. A hardware memory device having program instructions stored thereon that, upon execution by a processor of a Head-Mounted Display (HMD), cause the HMD to:
instruct a user wearing the HMD to perform a physical manipulation of an object identified in a video frame captured by a camera mounted on the HMD, the manipulation performed at a specified speed;
identify whether the user has performed the physical manipulation;
determine that the physical manipulation has been performed with the specified speed that is higher than a natural handling speed of the object; and
execute a security operation in response to the determination, wherein the security operation comprises at least one of:
(a) grant access to secured content in response to detecting the manipulation at the specified speed; or
(b) deny access to the secured content in response to detecting the manipulation at a different speed.

13. The hardware memory device of claim 12 , wherein the physical manipulation comprises a rotation or translation of the object by an amount set by the user during a training procedure.

14. The hardware memory device of claim 12 , wherein the program instructions, upon execution, further cause the HMD to display an image to the user highlighting the object.

15. The hardware memory device of claim 12 , wherein the image indicates a difference between: (i) a current rotation angle and a target rotation angle, or (ii) a current trajectory and a target trajectory during the physical manipulation.

16. The hardware memory device of claim 12 , wherein secured content is stored in an Information Handling System (IHS) coupled to the HMD.

17. A method, comprising:
recognizing an object based upon an image captured via a camera mounted on a Head-Mounted Display (HMD) during execution of an augmented reality (AR) application;
determining that: (a) the user clicked a key or uttered a command, and (b) after the click or utterance, the user manipulated the object with a specified speed; and
at least one of:
(a) granting access to secured content in response to: (i) the user having performed the manipulation at the specified speed, and (ii) detecting an absence of another person; or
(b) denying access to the secured content in response to: (i) the user having performed the manipulation at a different speed, and (ii) detecting a presence of another person.

Assignees

• Dell Products Lp

Inventors

• Hamlin Daniel L
• Yildiz Yagiz Can

Classifications

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V40/19

  Sensors therefor · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title