Systems and methods for a peripheral-centric augmented/virtual reality environment

Patent metadata

Field	Value
Publication number	US-10754416-B2
Application number	US-201815890161-A
Country	US
Kind code	B2
Filing date	Feb 6, 2018
Priority date	Nov 14, 2016
Publication date	Aug 25, 2020
Grant date	Aug 25, 2020

Abstract

In certain embodiments, a sensing and tracking system detects objects, such as user input devices or peripherals, and user interactions with them. A representation of the objects and user interactions are then injected into the virtual reality environment. The representation can be an actual reality, augmented reality, virtual representation or any combination. For example, an actual keyboard can be injected, but with the keys pressed being enlarged and lighted.

First claim

What is claimed is:

1. A method of operating a peripheral device, the method comprising:
receiving polling data from one or more sensors, the polling data corresponding to physical characteristics of a physical environment around the peripheral device;
determining an area to orient a projected virtual display relative to the peripheral device within the physical environment based on the physical characteristics;
determining a second area to orient a virtual interactive display relative to the peripheral device, wherein the interactive display is configured to facilitate an augmentation of functional capabilities of the peripheral device;
determining a spatial relationship between the peripheral device and the projected virtual display; and
generating control data configured to cause an AR/VR-based head-mounted display (HMD) to project the virtual display in the determined area and the determined second area at a maintained spatial relationship between the peripheral device and the projected virtual display as the peripheral device is moved within the physical environment.

2. The method of claim 1 further comprising:
detecting that the peripheral device is placed on a surface or interfaced by a user,
wherein receiving the polling data from the one or more sensors occurs in response to detecting that the peripheral device is placed on the surface or interfaced by the user.

3. The method of claim 1 wherein the control data causes the spatial relationship between the peripheral device and the virtual display to be maintained such that:
a movement of the peripheral device that is within a threshold distance from an initial location of the peripheral device does not cause the virtual display to move; and
a movement of the peripheral device that is greater than the threshold distance from the initial location of the peripheral device causes the spatial relationship between the peripheral device and the projected interactive display to be fixed,
the spatial relationship between the peripheral device and the projected interactive display is fixed as the peripheral device is moved in the physical environment.

4. The method of claim 1 further comprising:
determining a third area on the peripheral device to orient a virtual overlay, wherein the virtual overlay is configured to further facilitate the augmentation of the functional capabilities of the peripheral device; and
determining a spatial relationship between the peripheral device and the projected virtual overlay,
wherein the control data is further configured to cause the HIVID to project the virtual overlay in the determined third area and at a maintained spatial relationship between the peripheral device and the projected interactive display as the peripheral device is moved in the physical environment.

5. A system for operating a virtual workstation, the system comprising:
one or more processors;
one or more non-transitory computer-readable storage mediums containing instructions configured to cause the one or more processors to perform operations including:
receiving polling data from one or more sensors, the polling data corresponding to physical characteristics of a physical environment around a peripheral device;
determining an area to orient a projected virtual display relative to the peripheral device within the physical environment based on the physical characteristics;
determining a second area to orient a virtual interactive display relative to the peripheral device, wherein the virtual interactive display is configured to facilitate an augmentation of functional capabilities of the peripheral device;
determining a spatial relationship between the peripheral device and the projected virtual display; and
generating control data configured to cause an AR/VR-based head-mounted display (HMD) to project the virtual display in the determined area and the determined second area at a maintained spatial relationship between the peripheral device and the projected virtual display as the peripheral device is moved within the physical environment.

6. The system of claim 5 wherein the one or more non-transitory computer-readable storage mediums further contain instructions comprising:
detecting that the peripheral device is placed on a surface or interfaced by a user,
wherein receiving the polling data from the one or more sensors occurs in response to detecting that the peripheral device is placed on the surface or interfaced by the user.

7. The system of claim 5 wherein the control data causes the spatial relationship between the peripheral device and the virtual display to be maintained such that:
a movement of the peripheral device that is within a threshold distance from an initial location of the peripheral device does not cause the virtual display to move; and
a movement of the peripheral device that is greater than the threshold distance from the initial location of the peripheral device causes the spatial relationship between the peripheral device and the projected interactive display to be fixed,
the spatial relationship between the peripheral device and the projected interactive display is fixed as the peripheral device is moved in the physical environment.

8. The system of claim 5 wherein the one or more non-transitory computer-readable storage mediums further contain instructions comprising:
determining a third area on the peripheral device to orient a virtual overlay, wherein the virtual overlay is configured to further facilitate the augmentation of the functional capabilities of the peripheral device; and
determining a spatial relationship between the peripheral device and the projected virtual overlay,
wherein the control data is further configured to cause the HMD to project the virtual overlay in the determined third area and at a maintained spatial relationship between the peripheral device and the projected interactive display as the peripheral device is moved in the physical environment.

Assignees

• Logitech Europe Sa

Inventors

• Harvey Stephen
• O'Keeffe Denis
• Connellan Andreas
• O'Sullivan Damien
• Kehoe Aidan
• Curran Noirin
• Rouvinez Thomas
• Gutierrez Mario
• Riviere Olivier
• Zimmermann Remy
• Meisser Mathieu
• Onorio Dennin
• Trotman Ciaran
• O'Bradaigh Pierce
• Twohig Marcel
• Murphy Padraig
• Ahern Jerry

Classifications

• G06F3/0487

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06F3/016

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/014

  Hand-worn input/output arrangements, e.g. data gloves · CPC title