Systems and methods for enhancing augmented reality experience with dynamic output mapping

Patent metadata

Field	Value
Publication number	US-11210858-B2
Application number	US-201916687157-A
Country	US
Kind code	B2
Filing date	Nov 18, 2019
Priority date	Aug 24, 2015
Publication date	Dec 28, 2021
Grant date	Dec 28, 2021

Abstract

Disclosed herein are systems and methods for configuring a multi-device digital experience. In an embodiment, the method includes presenting a first content portion of a digital experience via a wearable display device in an environment; presenting a second content portion of the digital experience via a first device in a set of one or more devices in the environment; capturing test-signal-output data presented by the first device, the test-signal-output data being captured by the wearable display device in the environment, the test-signal-output data being based on a test signal sent to the first device; estimating spatial-characteristic data of the first device based on the captured test-signal-output data; and modifying the presentation of the second content portion of the digital experience via the first device based on the estimated spatial-characteristic data of the first device.

First claim

What is claimed is:

1. A method comprising:
initially presenting a first content portion of a digital augmented reality (AR) experience via a display of an AR display device in an environment that includes a set of one or more additional display devices, wherein the digital AR experience is initially presented in the environment only on the display of the AR display device;
sending a test signal to a first display device of the set of one or more additional display devices;
while the test signal is presented by the first display device, capturing test-signal-output data presented by the first display device, the test-signal-output data being based on the test signal;
after the initial presentation of the first content portion of the digital AR experience and during a continued presentation of the first content portion of the digital AR experience, extending the digital AR experience from only the AR display device to also the first display device such that a second content portion of the digital AR experience is being presented by the first display device in the environment;
estimating spatial-characteristic data of the first display device based on the captured test-signal-output data; and
modifying the second content portion of the digital AR experience based on at least one of the captured test-signal-output data and the estimated spatial-characteristic data of the first display device.

2. The method of claim 1 , further comprising sending, to the first display device, a second-content-portion stream that comprises the second content portion of the digital AR experience.

3. The method of claim 1 , wherein the test signal comprises a fiducial marker detectable by the AR display device.

4. The method of claim 1 , further comprising analyzing second-content-portion data presented by the first display device, wherein modifying the second content portion of the digital AR experience is further based on the analyzed second-content-portion data.

5. The method of claim 1 , wherein extending the digital AR experience from only the AR display device to also the first display device such that the second content portion of the digital AR experience is being presented via the first display device comprises:
sending, to the first display device, a second-content-portion stream that comprises the second content portion of the digital AR experience; and
extending the digital AR experience from only the AR display device to also the first display device and also a second display device in the set of one or more additional display devices in the environment, such that a third content portion of the digital AR experience is being presented via the second display device while the second content portion of the digital AR experience is being presented via the first display device.

6. The method of claim 5 , further comprising:
tracking a change in relative location of the AR display device with respect to the first display device in the environment and with respect to the second display device in the environment as the AR display device moves in the environment from a first location to a second location; and
modifying the second content portion and the third content portion of the digital AR experience based on the tracked change in relative location of the AR display device and based on the AR display device moving in the environment from the first location to the second location.

7. The method of claim 1 , wherein the AR display device comprises an RGB-D sensor, and wherein capturing the test-signal-output data further comprises capturing the test-signal-output data using the RGB-D sensor.

8. The method of claim 1 , further comprising iteratively adjusting the estimated spatial-characteristic data of the first device based on a comparison of the presented second content portion of the digital AR experience to an expected presentation of the second content portion of the digital AR experience.

9. The method of claim 1 , further comprising:
evaluating the presented second content portion of the digital AR experience based on an expected presentation of the second content portion of the digital AR experience; and
determining, based on the evaluating, whether the first display device is underperforming in presenting the second content portion of the digital AR experience.

10. The method of claim 1 , further comprising ceasing presentation of the second content portion of the digital AR experience responsive to determining that the first display device is underperforming in presenting the second content portion of the digital AR experience.

11. The method of claim 1 , wherein the estimated spatial-characteristic data of the first display device comprises at least one of location data of the first device or orientation data of the first display device.

12. The method of claim 1 , wherein:
estimating spatial-characteristic data of the first display device based on the captured test-signal-output data further comprises:
determining a physical distance between the AR display device and the first display device,
wherein modifying the second content portion of the digital AR experience based on the estimated spatial-characteristic data of the first display device further comprises:
in accordance with the determined physical distance, adding a black mask as the second content portion at the first display device to minimize an anticipated visual effect from the perspective of a user wearing the AR display device at the determined physical distance.

13. The method of claim 1 , further comprising using a dedicated content server to distribute content of the digital AR experience to the first display device.

14. A system comprising:
an augmented reality (AR) display device configured to:
initially present, via a display of the AR display device, a first content portion of a digital AR experience in an environment that includes a set of one or more additional display devices, wherein the digital AR experience is initially presented in the environment only on the display of the AR display device;
send a test signal to a first display device of the set of one or more additional display devices;
while the test signal is presented by the first display device, capture test-signal-output data presented by the first display device, the test-signal-output data being based on the test signal;
after the initial presentation of the first content portion of the digital AR experience and during a continued presentation of the first content portion of the digital AR experience, extend the digital AR experience from only the AR display device to also the first display device such that the first display device is configured to present a second content portion of the digital AR experience in the environment;
estimate spatial-characteristic data of the first display device based on the captured test-signal-output data; and
modify the second content portion of the digital AR experience based on at least one of the captured test-signal-output data and the estimated spatial-characteristic data of the first display device.

15. The system of claim 14 , wherein the AR display device is further configured to:
send, to the first display device, a second-content-portion stream that comprises the second content portion of the digital AR experience.

16. The system of claim 14 , wherein the test signal comprises a fiducial marker detectable by the AR display device.

17. The system of claim 14 , wherein the AR display device is further configured to iteratively adjust the estimated spatial-characteristic data of

Assignees

• Pcms Holdings Inc

Inventors

• Harviainen Tatu V J

Classifications

• G06F3/1423 Primary

  controlling a plurality of local displays, e.g. CRT and flat panel display · CPC title

• H04L67/131

  Protocols for games, networked simulations or virtual reality · CPC title

• H04S7/301

  Automatic calibration of stereophonic sound system, e.g. with test microphone · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title