Supporting an augmented-reality software application

Patent metadata

Field	Value
Publication number	US-2021287440-A1
Application number	US-201616338868-A
Country	US
Kind code	A1
Filing date	Nov 11, 2016
Priority date	Nov 11, 2016
Publication date	Sep 16, 2021
Grant date	—

Abstract

A computing device for supporting an Augmented-Reality (AR) software application is provided. The computing device is operative to select a physical location for placing a current virtual object, where the current virtual object appears to be placed when overlaid onto a video sequence capturing a physical scene in the surroundings of a user, based on an expected physical location which the user assumes in response to displaying the physical scene and the overlaid current virtual object to the user, and an attribute which is spatially dependent and which has an impact on a user experience of the AR software application. Taking the spatially-dependent attribute at the expected physical location of the user into consideration in selecting physical locations for placing virtual objects is advantageous in that virtual objects may be deployed at physical locations so as to provide an improved, or at least satisfactory, user experience to the users.

First claim

1 . A computing device for supporting an Augmented-Reality, AR, software application, the computing device comprising a processing circuitry being operative to:
select a physical location for placing a current virtual object, where the current virtual object appears to be placed when overlaid onto a video sequence capturing a physical scene in the surroundings of a user of the AR software application, based on:
an expected physical location to which the user moves, or where the user remains, in response to displaying the physical scene and the overlaid current virtual object to the user, and
an attribute which is spatially dependent in the surroundings of the user and which has an impact on a user experience of the AR software application.

2 . The computing device according to claim 1 , the processing circuitry being operative to select the physical location for placing the current virtual object by:
for at least one candidate physical location for placing the current virtual object:
determining the expected physical location which the user assumes in response to displaying the physical scene and the current virtual object, overlaid such that it appears to be placed at the candidate physical location, to the user, and
evaluating a value of the spatially-dependent attribute at the expected physical location, and
selecting the physical location for placing the current virtual object based on the plurality of values of the spatially-dependent attribute evaluated at the at least one candidate physical location.

3 . The computing device according to claim 2 , wherein the expected physical location which the user assumes in response to displaying the physical scene and the overlaid current virtual object to the user is within a predetermined range of the candidate physical location.

4 . The computing device according to claim 2 , wherein the expected physical location which the user assumes in response to displaying the physical scene and the overlaid current virtual object to the user is determined based on a type of the current virtual object.

5 . The computing device according to claim 4 , wherein the expected physical location which the user assumes in response to displaying the physical scene and the overlaid current virtual object to the user is determined based on information pertaining to expected behavior of users of the AR software application in response to displaying a virtual object of the same type as the current virtual object to the users.

6 . The computing device according to claim 4 , wherein the expected physical location which the user assumes in response to displaying the physical scene and the overlaid current virtual object to the user is determined based on a learned behavior of the user in response to previously displayed virtual objects of the same type as the current virtual object.

7 . The computing device according to claim 1 , being further operative to retrieve, from a database, one or more values of the spatially-dependent attribute as a function of a physical location in the surroundings of the user.

8 . The computing device according to claim 1 , the processing circuitry being further operative to store, in a database, measured values of the spatially-dependent attribute as a function of physical location.

9 . The computing device according to claim 7 , wherein the database is maintained by any one of: the computing device, a Radio Access Network, RAN, node, and an Operations Support Systems, OSS, node.

10 . The computing device according to claim 1 , wherein the spatially-dependent attribute is related to a performance of a wireless connection utilized by the AR software application.

11 . The computing device according to claim 10 , wherein the spatially-dependent attribute is any one of: a signal strength, a data rate, a bandwidth, an error rate, a retransmission rate, and a latency, of the wireless connection.

12 . The computing device according to claim 1 , wherein the spatially-dependent attribute is related to a perception by the user of sound which is rendered by the AR software application.

13 . The computing device according to claim 1 , the processing circuitry being further operative to overlay the current virtual object onto the video sequence capturing the physical scene, such that the overlaid current virtual object appears to be placed at the selected physical location.

14 . The computing device according to claim 13 , the processing circuitry being further operative to display the video sequence capturing the physical scene and the overlaid current virtual object to the user.

15 . The computing device according to claim 1 , wherein the video sequence is captured by a camera worn by the user, such that a field-of-view of the camera is correlated with a physical location and orientation of the user.

16 . The computing device according to claim 1 , wherein the display is worn by the user.

17 . The computing device according to claim 1 , being any one of: a gaming server hosting the AR software application, a virtual-reality headset, a head-mounted display, a mobile phone, a smartphone, a mobile terminal, a personal computer, a laptop, a tablet, and a gaming console.

18 . A method of supporting an Augmented-Reality, AR, software application, the method being performed by a computing device and comprising:
selecting a physical location for placing a current virtual object, where the current virtual object appears to be placed when overlaid onto a video sequence capturing a physical scene in the surroundings of a user of the AR software application, based on: an expected physical location to which the user moves, or where the user remains, in response to displaying the physical scene and the overlaid current virtual object to the user, and an attribute which is spatially dependent in the surroundings of the user and which has an impact on a user experience of the AR software application.

19 - 34 . (canceled)

35 . A computer program product comprising a non-transitory computer-readable storage medium, the computer-readable storage medium having embodied therein computer-executable instructions for causing a device to perform a method of supporting an Augmented-Reality, AR, software application, the method comprising:
selecting a physical location for placing a current virtual object, where the current virtual object appears to be placed when overlaid onto a video sequence capturing a physical scene in the surroundings of a user of the AR software application, based on:
an expected physical location to which the user moves, or where the user remains, in response to displaying the physical scene and the overlaid current virtual object to the user, and
an attribute which is spatially dependent in the surroundings of the user and which has an impact on a user experience of the AR software application.

Assignees

• Ericsson Telefon Ab L M

Inventors

• Wang Yu
• Bäckström Sandra
• Haraldson Johan
• Jeong Jaeseong
• Lawrenson Matthew John

Classifications

• G06F3/005 Primary

  Input arrangements through a video camera · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/04815

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title