Complementary augmented reality

Patent metadata

Field	Value
Publication number	US-2016371884-A1
Application number	US-201514742458-A
Country	US
Kind code	A1
Filing date	Jun 17, 2015
Priority date	Jun 17, 2015
Publication date	Dec 22, 2016
Grant date	—

Abstract

The described implementations relate to complementary augmented reality. One implementation is manifest as a system including a projector that can project a base image from an ancillary viewpoint into an environment. The system also includes a camera that can provide spatial mapping data for the environment and a display device that can display a complementary three-dimensional (3D) image to a user in the environment. In this example, the system can generate the complementary 3D image based on the spatial mapping data and the base image so that the complementary 3D image augments the base image and is dependent on a perspective of the user. The system can also update the complementary 3D image as the perspective of the user in the environment changes.

First claim

1 . A system, comprising:
a camera configured to provide spatial mapping data for an environment; a projector configured to project a base three-dimensional (3D) image from an ancillary viewpoint into the environment, the base 3D image being spatially-registered in the environment based at least in part on the spatial mapping data; a display configured to display a complementary 3D image to a user in the environment; and, a processor configured to:
generate the complementary 3D image based at least in part on the spatial mapping data and the base 3D image so that the complementary 3D image augments the base 3D image and is dependent on a perspective of the user, wherein the perspective of the user is different than the ancillary viewpoint, and
update the complementary 3D image as the perspective of the user in the environment changes.

2 . The system of claim 1 , further comprising a console that is separate from the projector and the display, wherein the console includes the processor.

3 . The system of claim 2 , further comprising another display configured to display another complementary 3D image to another user in the environment, wherein the processor is further configured to generate the another complementary 3D image that augments the base 3D image and is dependent on another perspective of the another user.

4 . The system of claim 1 , wherein the processor is further configured to determine the perspective of the user from the spatial mapping data.

5 . A system, comprising:
a projector configured to project a spatially-registered base three-dimensional (3D) image from a viewpoint into an environment; and, a display device configured to display a complementary 3D image that augments the base 3D image, the complementary 3D image being dependent on a perspective of a user in the environment.

6 . The system of claim 5 , further comprising a depth camera configured to provide spatial mapping data for the environment.

7 . The system of claim 6 , wherein the depth camera comprises multiple calibrated depth cameras.

8 . The system of claim 5 , wherein the viewpoint is an ancillary viewpoint and is different than the perspective of the user.

9 . The system of claim 5 , wherein the display device receives image data related to the base 3D image from the projector and the display device uses the image data to generate the complementary 3D image.

10 . The system of claim 5 , further comprising a processor configured to generate the complementary 3D image based at least in part on image data from the base 3D image and spatial mapping data of the environment.

11 . The system of claim 10 , wherein the viewpoint does not change with time and the processor is further configured to update the complementary 3D image as the perspective of the user changes with time.

12 . The system of claim 5 , further comprising additional projectors configured to project additional images from additional viewpoints into the environment.

13 . The system of claim 5 , further comprising another display device configured to display another complementary 3D image that augments the base 3D image and is dependent on another perspective of another user in the environment.

14 . The system of claim 5 , wherein the complementary 3D image is comprised of stereo images.

15 . The system of claim 5 , wherein the display device is further configured to display the complementary 3D image within a first field of view of the user, and wherein the projector is further configured to project the base 3D image such that a combination of the base 3D image and the complementary 3D image represents a second field of view of the user that is expanded as compared to the first field of view.

16 . The system of claim 15 , wherein the first field of view comprises a first angle that is less than 100 degrees and the second field of view comprises a second angle that is greater than 100 degrees.

17 . A device comprising:
a depth camera that collects depth data for an environment of a user; and, a complementary augmented reality component that determines a perspective of the user based at least in part on the depth data and generates a complementary three-dimensional (3D) image that is dependent on the perspective of the user, wherein the complementary 3D image augments a base 3D image that is projected into and spatially-registered in the environment, and wherein the complementary augmented reality component spatially registers the complementary 3D image in the environment based at least in part on the depth data.

18 . The device of claim 17 , further comprising a projector that projects the base 3D image.

19 . The device of claim 17 , wherein the complementary augmented reality component generates the base 3D image and spatially registers the base 3D image in the environment based at least in part on the depth data.

20 . The device of claim 17 , further comprising a display that displays the complementary 3D image such that the complementary 3D image partially overlaps the base 3D image.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Benko Hrvoje
• Wilson Andrew D
• Ofek Eyal
• Zheng Feng

Classifications

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B27/017

  Head mounted · CPC title

• G02B2027/0123

  comprising devices increasing the field of view · CPC title