Virtually representing spaces and objects while maintaining physical properties

Patent metadata

Field	Value
Publication number	US-10699491-B2
Application number	US-201916386934-A
Country	US
Kind code	B2
Filing date	Apr 17, 2019
Priority date	Jun 15, 2017
Publication date	Jun 30, 2020
Grant date	Jun 30, 2020

Abstract

Systems and techniques from displaying virtual representations of real-world spaces and objects in various environments are disclosed. A source environment at a first location can be scanned by a head-mounted display (HMD) device to generate three-dimensional datasets corresponding to the physical environment at the first location. The three-dimensional datasets can include detected physical properties associated with the physical environment. At a second location, the HMD can re-create the source environment, and render for display a virtual representation of the physical environment based on the three-dimensional datasets, where the virtual representation of the source environment is rendered to maintain any one of the detected physical properties associated with the physical environment. Further, at the second location, the HMD can enable a user to view one or more physical objects within the virtual representation of the physical environment such that the physical object is perceived to be within the source environment.

First claim

What is claimed is:

1. A computer storage medium storing computer-useable instructions that, when used by one or more computing devices, cause the one or more computing devices to perform operations comprising:
generating a first model of a first location based on input data obtained via an input device located at the first location;
segmenting the generated first model into a set of first model segments based on determined features of the generated first model;
receiving a second model of a physical object from a remote computing device located in a remote location, wherein the second model is generated based on additional input data obtained via another input device located at the remote location; and
providing for display the received second model positioned within the segmented first model.

2. The computer storage medium of claim 1 , the operations further comprising:
determining an illumination of the displayed first model; and
normalizing the displayed second model based on the determined illumination of the displayed first model.

3. The computer storage medium of claim 2 , wherein the displayed second model is normalized based further on one or more determined spatial relationships between the displayed second model and the displayed first model.

4. The computer storage medium of claim 1 , wherein the received second model is displayed at a scale that corresponds to the displayed first model.

5. The computer storage medium of claim 1 , the operations further comprising:
communicating the segmented first model and a relative position of the displayed second model to the remote computing device;
receiving a modified relative position associated with the displayed second model from the remote computing device based at least in part on the communicated relative position; and
updating the relative position of the displayed second model and the displayed first model based on the received modified relative position.

6. The computer storage medium of claim 1 , the operations further comprising:
modifying a relative position of a first model segment from the set of first model segments included in the displayed first model based on a detected input corresponding to the first model segment; and
communicating the modified relative position to the remote computing device, wherein the remote computing device is configured to provide for display the communicated first model having the first model segment at the modified relative position based on the communicated modified relative position.

7. The computer storage medium of claim 1 , the operations further comprising:
reducing dimensions of the generated first model and the received second model, wherein the received second model positioned within the generated first model is provided for display based on the reduced dimensions.

8. The computer storage medium of claim 7 , wherein the received second model and the generated first model is provided for display from a top-down view based on the reduced dimensions.

9. The computer storage medium of claim 1 , the operations further comprising:
completing each first model segment of the set of first model segments based on the determined features of the generated first model.

10. The computer storage medium of claim 9 , wherein each first model segment of the set of first model segments is completed based further on determined relationships between the first model segment and at least one other model segment of the set of first model segments.

11. The computer storage medium of claim 1 , the operations further comprising:
determining that a first set of dimensions associated with the received second model exceeds a second set of dimensions associated with the segmented first model, wherein the segmented first model is provided with display having a cutout portion that accommodates the received second model positioned within the displayed first model.

12. A computer-implemented method for transporting captured virtual environments, the method comprising:
generating, by a computing device, a first model of a first location based on input data obtained via an input device at the first location;
segmenting, by the computing device, the generated first model into at least a first model segment and a second model segment based on one or more determined features of the generated first model, wherein the first model segment corresponds to a physical object located in the first location;
receiving, by the computing device, a selection of the first model segment based on a received input that corresponds to the physical object; and
communicating, by the computing device, the selected first model segment to a remote computing device at a remote second location, wherein the remote computing device is configured to provide for display at least the communicated first model segment positioned within a segmented second model of the remote second location.

13. The computer-implemented method of claim 12 , wherein the second model is generated based on additional input data obtained via another input device at the remote second location.

14. The computer-implemented method of claim 12 , further comprising:
receiving, by the computing device, the segmented second model and a relative position of the displayed first model segment from the remote computing device; and
providing for display, by the computing device, the first model segment positioned within the received second model via a HMD coupled to the computing device based on the received relative position.

15. The computer-implemented method of claim 14 , wherein the first model segment positioned within the segmented second model is concurrently displayed via the HMD and another HMD coupled to the remote computing device.

16. The computer-implemented method of claim 14 , further comprising:
modifying, by the computing device, the received relative position of the displayed first model segment based on a detected input corresponding to the displayed first model segment; and
communicating, by the computing device, the modified relative position to the remote computing device, wherein the first model segment positioned within the segmented second model displayed via the other HMD is updated based on the communicated modified relative position.

17. The computer-implemented method of claim 14 , wherein the segmented second model includes a set of second model segments generated based on determined features of the generated second model, the method further comprising:
moving, by the computing device, a second model segment of the set of second model segments included in the displayed second model based on a detected interaction with the second model segment.

18. A system comprising:
a source environment scanning means for generating a model of a physical environment based on input data obtained via an input device located at the first location; and
a source environment segmenting means for segmenting the generated model into a set of model segments based on determined features of the generated model, wherein each model segment of the set of model segments segmented from the generated model corresponds to one of a set of physical objects located in the first location; and
a model communicating means for communicating at least one model segment, selected from the set of model segments segmented from the generated model, to a remote computing device based on a received input that corresponds to the at least one model segment.

19. The system of claim 18 , further comprising:
a model segm

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Pahud Michel
• Riche Nathalie
• Ofek Eyal
• Hurter Christophe

Classifications

• G06F3/03 Primary

  Arrangements for converting the position or the displacement of a member into a coded form · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title