Interactivity with a mixed reality

Patent metadata

Field	Value
Publication number	US-9600935-B2
Application number	US-201414184510-A
Country	US
Kind code	B2
Filing date	Feb 19, 2014
Priority date	Aug 29, 2005
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

Methods of interacting with a mixed reality are presented. A mobile device captures an image of a real-world object where the image has content information that can be used to control a mixed reality object through an offered command set. The mixed reality object can be real, virtual, or a mixture of both real and virtual.

First claim

What is claimed is:

1. A method of interacting with a mixed reality, comprising:
obtaining, by a mixed reality system, image data relating to a plurality of real-world objects;
identifying, by the mixed reality system, a first set of attributes associated with a first real-world object and a second set of attributes associated with a second real-world object from the plurality of real-world objects, based on the image data;
obtaining, by the mixed reality system, a first command set for a first real-world object based on the first set of attributes and a second command set for a second real-world object based on the second set of attributes from a distal database storing a plurality of command sets for multiple, mixed reality objects, wherein the first and second command sets comprise command addresses corresponding to commands that control functions of the first and the second real-world objects, respectively;
presenting, via a physical computer interface, a virtual interface comprising the first and the second command sets to a user, where at least one of a command from the first command set and a command from the second command set is superimposed on an image of a corresponding real-world interface part of at least one of the corresponding first real-world object and second real-world object, and is configured to stay in proper relation to the at least one of the corresponding first real-world object and second real-world object within a mixed reality view, and wherein the at least one command controls the function associated with the corresponding real-world interface part; and
executing, by the mixed reality system, the at least one command from at least one of the first and second command sets upon the user interacting directly with the image of the corresponding real-world interface part by issuing a call to the corresponding command address of the at least one command.

2. The method of claim 1 , wherein at least one of the first real-world object and the second real-world object is an appliance, a kitchen appliance, a business office appliance, a clinic appliance, a printer, a set-top box, a television, a vending machine, a radio, and a stereo.

3. The method of claim 1 , wherein the first real-world object is physically distinct from the second real-world object.

4. The method of claim 3 , wherein the first real-world object and second real-world object comprise objects from the same class.

5. The method of claim 3 , wherein the first real-world object is part of a first mixed-reality object and the second real-world object is part of a second mixed-reality object.

6. The method of claim 1 , wherein at least one of the first real-world object and the second real-world object is a component of a consumer electronic appliance.

7. The method of claim 6 , wherein the command is one of an “on” command and an “off” command.

8. The method of claim 1 wherein, the first real-world object and the second real-world object are parts of a multi-part object.

9. The method of claim 1 , wherein the attributes comprises at least one of the following an observed state, a name, a class, and an environmental characteristic.

10. The method of claim 1 , wherein the execution of the command results in changing a channel.

11. The method of claim 1 , wherein the execution of the command results in playing music.

12. The method of claim 1 , wherein the image data comprises a still image.

13. The method of claim 1 , wherein the image data comprises a video image.

14. The method of claim 1 , further comprising using a sensor associated with the mixed reality system.

15. The method of claim 14 , wherein the sensor provides information associated with a physical location.

16. The method of claim 14 , wherein the sensor provides information associated with at least one of an orientation and an acceleration.

17. The method of claim 1 , wherein the first and second command sets comprise non-visible command sets.

Assignees

• Nant Holdings Ip Llc

Inventors

• Cohen Ronald H

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• H04M2250/52

  including functional features of a camera · CPC title

• H04N21/4223 Primary

  Cameras (H04N23/00 takes precedence) · CPC title

• G06T19/00

  Manipulating three-dimensional [3D] models or images for computer graphics · CPC title

• A63F13/52 Primary

  involving aspects of the displayed game scene · CPC title