Protecting privacy in web-based immersive augmented reality

Patent metadata

Field	Value
Publication number	US-9679144-B2
Application number	US-201314082051-A
Country	US
Kind code	B2
Filing date	Nov 15, 2013
Priority date	Nov 15, 2013
Publication date	Jun 13, 2017
Grant date	Jun 13, 2017

Abstract

An “AR Privacy API” provides an API that allows applications and web browsers to use various content rendering abstractions to protect user privacy in a wide range of web-based immersive augmented reality (AR) scenarios. The AR Privacy API extends the traditional concept of “web pages” to immersive “web rooms” wherein any desired combination of existing or new 2D and 3D content is rendered within a user's room or other space. Advantageously, the AR Privacy API and associated rendering abstractions are useable by a wide variety of applications and web content for enhancing the user's room or other space with web-based immersive AR content. Further, the AR Privacy API is implemented using any existing or new web page coding platform, including, but not limited to HTML, XML, CSS, JavaScript, etc., thereby enabling existing web content and coding techniques to be smoothly integrated into a wide range of web room AR scenarios.

First claim

What is claimed is:

1. A computer-implemented process for using a web browser to render content in a 3D space, comprising:
using a computer to perform process actions for:
evaluating data from a plurality of sensors to infer environmental information regarding a space and regarding persons and contents within that space, the environmental information comprising any combination of geometric characteristics of the space and geometric characteristics and positions of contents and persons within the space;
evaluating the environmental information to identify a plurality of surfaces in the space upon which content can be rendered by one or more output devices;
labeling one or more of the surfaces with semantic annotations based in part on the environmental information, each semantic annotation comprising one or more physical characteristics and a physical location of a corresponding one of the surfaces;
receiving content via a networking device from any combination of websites, web pages and web rooms, the content further comprising one or more abstractions that declaratively specify how that content is to be adapted relative to any of a plurality of semantic annotations;
evaluating the abstractions in combination with the semantic annotations associated with the surfaces to determine one or more corresponding surfaces on which the content is to be rendered; and
applying the abstractions and corresponding semantic annotations to adapt and render the content onto the corresponding surfaces, via one or more output devices, without disclosing the environmental information to any of the websites, web pages and web rooms.

2. The computer-implemented process of claim 1 further comprising steps for associating a permission state with one or more elements of the environmental information.

3. The computer-implemented process of claim 2 wherein one or more of the permission states are individually granted to particular websites, web pages and web rooms.

4. The computer-implemented process of claim 2 wherein one or more of the permission states are granted to multiple websites, web pages and web rooms.

5. The computer-implemented process of claim 1 wherein the abstractions are interpreted by a trusted renderer that receives content from one or more of the websites, web pages and web rooms and determines which surface that content is to be rendered onto without allowing the websites, web pages and web rooms to obtain information regarding that surface.

6. The computer-implemented process of claim 5 wherein a JavaScript-based API provides an interface between the trusted renderer and each websites, web pages and web rooms.

7. The computer-implemented process of claim 1 wherein a constraint solver is used to evaluate the environmental data to determine the surfaces in the space upon which content is to be rendered.

8. The computer-implemented process of claim 7 wherein the constraint solver evaluates the environmental data and the abstractions of the content relative to the semantic annotations associated with corresponding surfaces to identify which of the surfaces are to be used to render particular content.

9. The computer-implemented process of claim 1 wherein the abstractions further define relative positional relationships between two or more particular segments of the content.

10. A system for rendering content to construct an augmented reality environment, comprising:
a device for evaluating a plurality of sensors for inferring environmental information regarding a space and regarding persons and contents within that space, the environmental information comprising any combination of geometric characteristics of the space and geometric characteristics and positions of contents and persons within the space;
a device for evaluating the environmental information to identify a plurality of surfaces in the space upon which content can be rendered;
a device for associating one or more semantic annotations with one or more of the surfaces, each semantic annotation comprising one or more physical characteristics and a physical location of a corresponding one of the surfaces;
a networking device for receiving content from one or more websites, web pages and web rooms, the content further comprising one or more abstractions that declaratively specify how the content is to be adapted relative to any of a plurality of semantic annotations;
a device for interpreting the abstractions of the content to associate that content with one or more corresponding surfaces based on the semantic annotations associated with those surfaces; and
an output device, comprising any combination of monitors, projectors, touchscreens, tablets, phones, and media players, for rendering the content onto the corresponding surfaces based on the corresponding abstractions and semantic annotations without disclosing corresponding environmental information regarding the surfaces to the websites, web pages or web rooms that were the source of the content.

11. The system of claim 10 further comprising steps for associating a permission state with one or more elements of the environmental information.

12. The system of claim 11 wherein one or more of the permission states are individually granted to particular sources of the content.

13. The system of claim 11 wherein one or more of the permission states are granted to multiple sources of the content.

14. The system 10 wherein a JavaScript-based API provides a trusted interface between the device for rendering content and the source of that content.

15. The system of claim 10 wherein a constraint solver is used to evaluate the environmental data to identify the surfaces relative to any semantic annotations associated with any corresponding surface to associate specific content with one or more corresponding surfaces based on one or more of the abstractions of that content.

16. The system of claim 10 wherein the abstractions further define relative positional relationships between two or more segments of the content.

17. A computer-readable device having computer executable instructions stored therein for rendering content in a 3D space, said instructions causing a computing device to execute a method comprising:
evaluating data from a plurality of sensors to infer environmental information regarding surfaces, objects and persons within a 3D space, the environmental information comprising any combination of geometric characteristics of the space and geometric characteristics and positions of objects and persons within the space;
evaluating the environmental information to identify a plurality of surfaces in the 3D space upon which content can be rendered;
associating one or more semantic annotations with one or more of the surfaces, each semantic annotation comprising one or more physical characteristics and a physical location of a corresponding one of the surfaces;
receiving content from one or more websites, web pages and web rooms via a communications device, the content further comprising one or more abstractions that declaratively specify how the content is to be adapted relative to one or more of a plurality of semantic annotations;
applying a constraint solver to evaluate the environmental data and the semantic annotations associated with any corresponding surface to associate specific content with one or more corresponding surfaces based on the abstractions of that content; and
applying one or more output devices for rendering the content onto the corresponding surfaces based on the corresponding abstractions and semantic annotations without disclosing

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Molnar David
• Vilk John
• Ofek Eyal
• Moshchuk Alexander
• Wang Jiahe
• Gal Ran
• Shapira Lior
• Burger Douglas Christopher
• Macintyre Blair
• Livshits Benjamin

Classifications

• G06F21/6245 Primary

  Protecting personal data, e.g. for financial or medical purposes · CPC title

• H04L67/38

  Electricity · mapped topic

• H04L67/12

  specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks · CPC title

• G06F21/629

  to features or functions of an application · CPC title

• H04L67/02

  based on web technology, e.g. hypertext transfer protocol [HTTP] · CPC title