Multi-user content sharing in immersive virtual reality environments

Patent metadata

Field	Value
Publication number	US-12050757-B2
Application number	US-202318114156-A
Country	US
Kind code	B2
Filing date	Feb 24, 2023
Priority date	Dec 18, 2014
Publication date	Jul 30, 2024
Grant date	Jul 30, 2024

Abstract

The technology disclosed relates to user interfaces for controlling augmented reality environments. Real and virtual objects can be seamlessly integrated to form an augmented reality by tracking motion of one or more real objects within view of a wearable sensor system using a combination a RGB (red, green, and blue) and IR (infrared) pixels of one or more cameras. It also relates to enabling multi-user collaboration and interaction in an immersive virtual environment. In particular, it relates to capturing different sceneries of a shared real world space from the perspective of multiple users. The technology disclosed further relates to sharing content between wearable sensor systems. In particular, it relates to capturing images and video streams from the perspective of a first user of a wearable sensor system and sending an augmented version of the captured images and video stream to a second user of the wearable sensor system.

First claim

What is claimed is:

1. A method of sharing content between wearable sensor systems, the method including:
capturing a first video stream of a first portion of a real world space using at least one camera electronically coupled to a first wearable sensor system;
generating, for display towards a first user, a first presentation output based on information from the first video stream and corresponding to the first portion of the real world space;
capturing a second video stream of a second portion of the real world space using at least one camera electronically coupled to the first wearable sensor system, wherein the first video stream and the second video stream correspond to a concurrent timespan;
sending, in association with a communication channel established between the first wearable sensor system and a second wearable sensor system, the second video stream to the second wearable sensor system; and
generating, for display towards a second user, a second presentation output based on information from the second video stream and corresponding to the second portion of the real world space.

2. The method of claim 1 , wherein a first set of features is detected corresponding to the first video stream and a second set of features is detected corresponding to the second video stream, the first set of features being processed to generate the first presentation output displayed towards the first user and the second set of features being processed to generate the second presentation output displayed towards the second user.

3. The method of claim 2 , wherein the second set of features further comprises one or more overlapping features in relation to the first set of features, and one or more nonoverlapping features in relation to the second set of features, the one or more nonoverlapping features relating to at least one of:
(i) a field of view corresponding to the second video stream,
(ii) a resolution of the second video stream,
(iii) a luminance measure of the second video stream,
(iv) an additional virtual graphic overlay to be combined with the second video stream, and
(v) an additional annotation overlay to be combined with the second video stream.

4. The method of claim 3 , wherein the first presentation output comprises a first virtual interface associated with a virtual reality system, and wherein the second presentation output comprises a second virtual interface associated with the virtual reality system.

5. The method of claim 4 ,
wherein at least one of the first feature set and the second feature set is processed, as input, to generate, as output, a three-dimensional map of at least one of the first portion and the second portion of the real world space, and
wherein the generated three-dimensional map is presented to a virtual reality system and the virtual reality system applies the generated three-dimensional map as a constraint upon which to construct a virtual reality environment.

6. The method of claim 5 ,
wherein the first virtual interface, responsive to a selection of an operation by the first user, enables the first user to interact with content within the virtual reality environment, and
wherein the second virtual interface, responsive to a selection of an operation by the second user, enables the second user to interact with content within the virtual reality environment.

7. The method of claim 6 , wherein the virtual reality environment enables collaboration between the first user and the second user participating in a collaborative operation, the collaborative operation facilitated by the first virtual interface and the second virtual interface, respective to the first user and the second user.

8. The method of claim 5 , wherein the virtual reality environment is configured to be updated to display one or more of new content and altered content, the updating of the display further comprising:
receiving, from the first wearable sensor system, an updated video stream of at least one of the first portion of the real world space and second portion of the real world space;
detecting, from the updated video stream, an updated set of features corresponding to the updated video stream;
processing, as input, at least one of the updated feature set and a previous three-dimensional map to generate, as output, an updated three-dimensional map; and
presenting, to the virtual reality system, the updated three-dimensional map to be applied as a constraint on the virtual reality environment.

9. The method of claim 1 , further comprising:
capturing a third video stream of a third portion of the real world space using at least one camera electronically coupled to the second wearable sensor system, wherein the first video stream and the third video stream correspond to a concurrent timespan;
generating, for display towards a second user, a third presentation output based on information from the third video stream and corresponding to the third portion of the real world space; and
sharing, in association with the communication channel established between the first wearable sensor system and the second wearable sensor system, data associated with the first presentation output and data associated with the third presentation output to at least one of the first wearable sensor system and the second wearable sensor system.

10. The method of claim 1 , wherein the first portion of the real world space and the second portion of the real world space overlap one another.

11. The method of claim 1 , wherein the first portion of the real world space and the second portion of the real world space do not overlap one another.

12. The method of claim 1 , wherein the first portion of the real world space is completely overlapped by the second portion of the real world space.

13. A non-transitory computer-readable recording medium having computer instructions recorded thereon for sharing content between wearable sensor systems, the computer instructions, when executed on one or more processors, causing the one or more processors to implement operations comprising:
capturing a first video stream of a first portion of a real world space using at least one camera electronically coupled to a first wearable sensor system;
generating, for display towards a first user, a first presentation output based on information from the first video stream and corresponding to the first portion of the real world space;
capturing a second video stream of a second portion of the real world space using at least one camera electronically coupled to the first wearable sensor system, wherein the first video stream and the second video stream correspond to a concurrent timespan;
sending, in association with a communication channel established between the first wearable sensor system and a second wearable sensor system, the second video stream to the second wearable sensor system; and
generating, for display towards a second user, a second presentation output based on information from the second video stream and corresponding to the second portion of the real world space.

14. The non-transitory computer-readable recording medium of claim 13 , wherein a first set of features is detected corresponding to the first video stream and a second set of features is detected corresponding to the second video stream, the first set of features being processed to generate the first presentation output displayed towards the first user and the second set of features being processed to generate the second presentation output displayed towards the second user.

15. The non-transitory computer-readable recording medium of claim 14 ,
wherein at least o

Assignees

• Ultrahaptics Ip Two Ltd

Inventors

• Holz David S
• Fox Barrett
• Hay Kyle A
• Hare Gabriel A
• Yu Wilbur Yung Sheng
• Edelhart Dave
• Medich Jody
• Plemmons Daniel

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title

• G06F3/04847

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title