Augmentation modification based on user interaction with augmented reality scene

Patent metadata

Field	Value
Publication number	US-2016284134-A1
Application number	US-201514667302-A
Country	US
Kind code	A1
Filing date	Mar 24, 2015
Priority date	Mar 24, 2015
Publication date	Sep 29, 2016
Grant date	—

Abstract

Apparatuses, methods, and storage media for modifying augmented reality in response to user interaction are described. In one instance, the apparatus for modifying augmented reality may include a processor, a scene capture camera coupled with the processor to capture a physical scene, and an augmentation management module to be operated by the processor. The augmentation management module may obtain and analyze the physical scene, generate one or more virtual articles to augment a rendering of the physical scene based on a result of the analysis, track user interaction with the rendered augmented scene, and modify or complement the virtual articles in response to the tracked user interaction. Other embodiments may be described and claimed.

First claim

What is claimed is:

1 . An apparatus for providing augmented reality computing, comprising:
a processor; a scene capture camera coupled with the processor to capture a physical scene; and an augmentation management module to be operated by the processor to:
obtain and analyze the physical scene,
generate one or more virtual articles to augment a rendering of the physical scene, based on a result of the analysis,
track user interaction with the rendered augmented scene, and
modify or complement the one or more virtual articles, in response to the tracked user interaction.

2 . The apparatus of claim 1 , further comprising: an interaction capture camera coupled with the processor to capture the user interaction with the augmented scene and to provide information about the captured interaction to the augmentation management module for tracking.

3 . The apparatus of claim 2 , wherein the apparatus is selected from one of: a laptop computing device, a tablet computing device, a mobile computing device, or an all-in-one (AIO) computing device.

4 . The apparatus of claim 1 , wherein the augmentation management module to track user interaction with the rendered augmented scene includes to obtain an indication of user interaction with at least one of the one or more virtual articles in the rendered augmented scene.

5 . The apparatus of claim 4 , wherein the augmentation management module to modify or complement the rendered augmented scene includes to:
align the at least one of the one or more virtual articles in the rendered augmented scene with the indicated user interaction with the at least one of the one or more virtual articles in the rendered augmented scene; alter a position of the at least one of the one or more virtual articles in the rendered augmented scene in response to the indication of the user interaction with the at least one of the one or more virtual articles in the rendered augmented scene; or alter the at least one of the one or more virtual articles in the rendered augmented scene in response to the indication of the user interaction with the at least one of the one or more virtual articles in the rendered augmented scene.

6 . The apparatus of claim 4 , wherein the indication of user interaction includes at least a selected one of: a gesture, a change of facial expression, a verbal command, a change in eye gaze, a change of posture, or a change of a head pose.

7 . The apparatus of claim 1 , further comprising: a display device coupled with the processor to display the rendered augmented scene to the user.

8 . The apparatus of claim 1 , wherein the augmentation management module to augment a rendering of the physical scene with one or more virtual articles includes to augment the rendering based at least in part on one or more markers associated with the physical scene.

9 . The apparatus of claim 1 , wherein the augmentation management module is to modify or complement the augmented scene substantially simultaneously with the tracking of the user interaction.

10 . The apparatus of claim 2 , wherein each of the cameras comprises a two-dimensional (2D) or three-dimensional (3D) camera to capture a real-time depth data and color data associated with the physical scene or the user interaction respectively, wherein the color data includes red, green, and blue (RGB) data.

11 . The apparatus of claim 10 , wherein the apparatus is placed on a substantially horizontal surface, to enable the capture of the physical scene and the user interaction.

12 . A computer-implemented method for providing augmented reality computing, comprising:
augmenting, by a computing device, a rendering of a physical scene with one or more virtual articles; tracking, by the computing device, user interaction with the rendered augmented scene, and modifying or complementing, by the computing device, the one or more virtual articles in the rendered augmented scene in response to the tracking of the user interaction.

13 . The computer-implemented method of claim 12 , further comprising:
obtaining and analyzing, by the computing device, the physical scene; and generating, by the computing device, the one or more virtual articles to augment the rendering of the physical scene, based on the analyzing the physical scene.

14 . The computer-implemented method of claim 12 , further comprising:
rendering, by the computing device, the augmented scene for display.

15 . The computer-implemented method of claim 14 , wherein tracking user interaction with the rendered augmented scene includes:
obtaining, by the computing device, an indication of user interaction with at least one of the one or more virtual articles in the rendered augmented scene, and wherein modifying or complementing the one or more articles further includes: aligning, by the computing device, the at least one of the one or more virtual articles in the rendered augmented scene with the indicated user interaction substantially simultaneously with and in response to obtaining of indication of the user interaction with the at least one of the one or more virtual articles in the rendered augmented scene.

16 . The computer-implemented method of claim 15 , wherein obtaining an indication of user interaction includes detecting, by the computing device, at least a selected one of: a gesture, a change of facial expression, a verbal command, a change in eye gaze, a change of posture, or a change of a head pose.

17 . One or more computer-readable media having instructions for providing augmented reality computing stored thereon that, in response to execution by a computing device, provide the computing device with an augmentation management environment to:
augment a rendering of a physical scene with one or more virtual articles; track user interaction with the rendered augmented scene, and modify or complement the one or more articles in the rendered augmented scene in response to the tracking of the user interaction.

18 . The computer-readable media of claim 17 , wherein the computing device is further provided with the augmentation management environment to:
obtain information about the physical scene; analyze information about the physical scene; and generate the one or more virtual articles to augment the rendering of the physical scene based on a result of the analysis of the physical scene.

19 . The computer-readable media of claim 17 , wherein the computing device is provided with the augmentation management environment to track user interaction with the rendered augmented scene includes the augmentation management environment to:
obtain an indication of user interaction with at least one of the one or more virtual articles in the rendered augmented scene.

20 . The computer-readable media of claim 19 , wherein the computing device is provided with the augmentation management environment to modify or complement the one or more articles includes the augmentation management environment to align the at least one of the one or more virtual articles in the scene with the indicated user interaction substantially simultaneously with and in response to the obtaining of the indication of user interaction with the at least one of the one or more virtual articles.

Assignees

• Intel Corp

Inventors

• Kamhi Gila
• Moran Amit

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/005 Primary

  Input arrangements through a video camera · CPC title

• G06T19/20 Primary

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title