Augmentation modification based on user interaction with augmented reality scene

Patent metadata

Field	Value
Publication number	US-10488915-B2
Application number	US-201715786035-A
Country	US
Kind code	B2
Filing date	Oct 17, 2017
Priority date	Mar 24, 2015
Publication date	Nov 26, 2019
Grant date	Nov 26, 2019

Abstract

Apparatuses, methods, and storage media for modifying augmented reality in response to user interaction are described. In one instance, the apparatus for modifying augmented reality may include a processor, a scene capture camera coupled with the processor to capture a physical scene, and an augmentation management module to be operated by the processor. The augmentation management module may obtain and analyze the physical scene, generate one or more virtual articles to augment a rendering of the physical scene based on a result of the analysis, track user interaction with the rendered augmented scene, and modify or complement the virtual articles in response to the tracked user interaction. Other embodiments may be described and claimed.

First claim

What is claimed is:

1. A mobile computing device, comprising:
at least one camera to capture one or more images of a physical scene including one or more physical objects in an environment of the mobile computing device;
at least one screen to display an augmented scene representative of the physical scene;
at least one storage device to store instructions; and
at least one processor to execute the instructions to:
generate a virtual object to display on the at least one screen within the augmented scene to simulate a presence of the virtual object within the environment; and
simulate at least one of a first manipulation type or a second manipulation type of the virtual object within the augmented scene based on data representative of a corresponding pointing gesture or a waiving gesture of a user hand in the environment.

2. The mobile computing device as defined in claim 1 , wherein the processor is to simulate the first manipulation type of the virtual object by causing the virtual object to move in response to detecting the pointing gesture of the user hand in the environment based on the one or more images.

3. The mobile computing device as defined in claim 1 , wherein the at least one processor is to track the pointing gesture or the waiving gesture of the user hand interacting with the virtual object based on the one or more images.

4. The mobile computing device as defined in claim 1 , wherein the mobile computing device includes at least one of a laptop, a tablet or a mobile telephone.

5. The mobile computing device as defined in claim 1 , wherein the at least one processor is to simulate the at least one of the first or second manipulation type of the virtual object in response to at least one of a change of facial expression, a verbal command, a change in eye gaze, a change in posture, or a change of a head position.

6. The mobile computing device as defined in claim 1 , wherein the at least one processor is to generate the virtual object in response to detection of one or more markers associated with the physical scene.

7. The mobile computing device as defined in claim 1 , further including one or more sensors to generate data associated with at least one of the environment or the mobile computing device.

8. The mobile computing device as defined in claim 7 , wherein the one or more sensors include at least one of a proximity sensor, an inertial sensor, an optical sensor, an audio sensor, a temperature sensor, a motion sensor, or a microphone.

9. At least one computer-readable storage device or storage disk comprising machine-readable instructions that, when executed, cause at least one processor to, at least:
cause at least one camera to capture one or more images of a physical scene, the physical scene including one or more physical objects in an environment of a mobile computing device;
cause at least one screen to display an augmented scene representative of the physical scene;
generate a virtual object to display on the at least one screen within the augmented scene to simulate presence of the virtual object within the environment; and
simulate at least one of a first manipulation type or a second manipulation type of the virtual object within the augmented scene based on data representative of a corresponding pointing gesture or a waiving gesture of a user hand in the environment.

10. The at least one computer-readable medium storage device or storage disk as defined in claim 9 , wherein the machine-readable instructions, when executed, cause the at least one processor to move the virtual object in response to detecting the pointing gesture of the user hand in the environment.

11. The at least one computer-readable storage device or storage disk as defined in claim 9 , wherein the machine-readable instructions, when executed, cause the at least one processor to track the pointing gesture or the waiving gesture of the user hand interacting with the virtual object.

12. The at least one computer-readable storage device or storage disk as defined in claim 9 , wherein the machine-readable instructions, when executed, cause the at least one processor to simulate the at least one of the first or second manipulation type of the virtual object in response to at least one of a change of facial expression, a verbal command, a change in eye gaze, a change in posture, or a change of a head position.

13. The at least one computer-readable storage device or storage disk as defined in claim 9 , wherein the machine-readable instructions, when executed, cause the at least one processor to generate the virtual object in response to one or more markers associated with the physical scene.

14. The at least one computer-readable storage device or storage disk as defined in claim 9 , wherein the machine-readable instructions, when executed, cause the at least one processor to generate data from one or more sensors, the generated data associated with the environment.

15. A mobile computing device to manipulate a virtual object, the mobile computing device comprising:
means for capturing one or more images of a physical scene, the physical scene including one or more physical objects in an environment of the mobile computing device;
means for displaying an augmented scene representative of the physical scene; and
means for generating the virtual object to display on the displaying means, the generating means to cause the virtual object to be presented within the augmented scene to simulate presence of the virtual object within the environment, the generating means to simulate at least one of a first manipulation type or a second manipulation type of the virtual object within the augmented scene based on data representative of a corresponding pointing gesture or a waiving gesture of a hand in the environment.

16. The mobile computing device as defined in claim 15 , wherein the generating means is to cause the virtual object to move in response to the pointing gesture of the user hand in the environment.

17. The mobile computing device as defined in claim 15 , further including means for tracking the at least one of the pointing gesture or the waiving gesture of the hand to facilitate simulating interaction with the virtual object.

18. The mobile computing device as defined in claim 15 , wherein the mobile computing device includes at least one of a laptop, a tablet or a mobile telephone.

19. The mobile computing device as defined in claim 15 , wherein the generating means is to simulate the at least one of the first or second manipulation type of the virtual object in response to at least one of a change of facial expression, a verbal command, a change in eye gaze, a change in posture, or a change of a head position.

20. The mobile computing device as defined in claim 15 , wherein the generating means is to generate the virtual object in response to one or more markers associated with the physical scene.

21. The mobile computing device as defined in claim 15 , further including means for sensing an object in the environment.

22. The mobile computing device as defined in claim 21 , wherein the sensing means includes at least one of a proximity sensor, an inertial sensor, an optical sensor, an audio sensor, a temperature sensor, a motion sensor, or a microphone.

Assignees

• Intel Corp

Inventors

• Kamhi Gila
• Moran Amit

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/005 Primary

  Input arrangements through a video camera · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title