Interactive augmented reality using a self-propelled device

Patent metadata

Field	Value
Publication number	US-9827487-B2
Application number	US-201314054636-A
Country	US
Kind code	B2
Filing date	Oct 15, 2013
Priority date	May 14, 2012
Publication date	Nov 28, 2017
Grant date	Nov 28, 2017

Abstract

A method is disclosed for operating a mobile computing device. The method may include a communication link between the mobile computing device and a second computing device. The second computing device may provide a virtual environment for the mobile computing device. Furthermore, the mobile computing device may allow a user to control a self-propelled device, which may be rendered as a virtual entity upon the virtual environment.

First claim

What is claimed is:

1. A method for operating a mobile computing device to control a remote controlled vehicle, the method being performed by one or more processors of the mobile computing device and comprising:
in response to user inputs on a user interface generated on a display of the mobile computing device, transmitting control information to the remote controlled vehicle to control movement of the remote controlled vehicle in a real-world environment;
interfacing with a second computing device that provides, via execution of a program by one or more processors of the second computing device, a virtual environment;
transmitting position information, corresponding to the remote controlled vehicle, to the second computing device to cause the second computing device to render a virtual entity upon the virtual environment, the virtual entity representing the remote controlled vehicle moving in the real-world environment;
receiving, from the second computing device, data corresponding to the virtual entity within the virtual environment; and
displaying, on the display of the mobile computing device, the virtual environment including the virtual entity representing the remote controlled vehicle.

2. The method of claim 1 , further comprising:
detecting an event relating to the remote controlled vehicle in the real-world environment;
determining a virtual event based on the detected event; and
incorporating the virtual event into the virtual environment.

3. The method of claim 1 , further comprising:
accessing saved data associated with the virtual environment stored on the second computing device, the saved data corresponding to previous interactions within the virtual environment; and
congruently linking the saved data to a current control session comprising the virtual environment and control of the remote controlled vehicle in the real-world environment.

4. The method of claim 3 , wherein the previous interactions and the current control session of the remote controlled vehicle correspond to gameplay, simultaneously controlling the remote controlled vehicle in the real-world environment and the virtual entity within the virtual environment.

5. The method of claim 4 , wherein the gameplay is associated with one or more of a minigame, emergent gameplay, or a mobile game.

6. The method of claim 1 , wherein the virtual environment displayed on the mobile computing device is linked to a display of the virtual environment through the second computing device.

7. The method of claim 1 , wherein the virtual environment corresponds to augmented reality.

8. The method of claim 1 , wherein the mobile computing device is one or more of a smart phone, a tablet computer, or a laptop computer.

9. The method of claim 1 , wherein interfacing with the second computing device is performed by the mobile computing device.

10. The method of claim 1 , wherein the step of interfacing is performed by the second computing device.

11. A system comprising:
a remote controlled vehicle; and
a mobile computing device comprising one or more processors executing a set of instructions that cause the one or more processors to:
in response to user inputs on a user interface generated on a display of the mobile computing device, transmit control information to the remote controlled vehicle to control movement of the remote controlled vehicle in a real-world environment;
interface with a second computing device that provides, via execution of a program by one or more processors of the second computing device, a virtual environment;
transmit position information, corresponding to the remote controlled vehicle, to the second computing device to cause the second computing device to render a virtual entity upon the virtual environment, the virtual entity representing the remote controlled vehicle moving in the real-world environment;
receive, from the second computing device, data corresponding to the virtual entity within the virtual environment; and
display, on the display of the mobile computing device, the virtual environment including the virtual entity representing the remote controlled vehicle.

12. The system of claim 11 , wherein executed set of instructions further cause the one or more processors of the mobile computing device to:
detect an event associated with the remote controlled vehicle in the real-world environment;
determine a virtual event based on the detected event; and incorporate the virtual event in the virtual environment.

13. The system of claim 11 , wherein the executed set of instructions further cause the one or more processors of the mobile computing device to:
access saved data associated with the virtual environment stored on the second computing device, the saved data corresponding to previous interactions within the virtual environment.

14. The system of claim 13 , wherein the executed set of instructions further cause the one or more processors of the mobile computing device to:
congruently link the saved data to a current control session comprising the virtual environment and control of the remote controlled vehicle in the real-world environment.

15. The system of claim 14 , wherein the previous interactions and the current control session correspond to gameplay within the virtual environment.

16. The system of claim 15 , wherein the gameplay is associated with one or more of a minigame, emergent gameplay, or a mobile game.

17. The system of claim 11 , wherein the virtual environment corresponds to augmented reality.

18. The system of claim 11 , wherein the mobile computing device is one or more of a smart phone, a tablet computer, or a laptop computer.

19. The system of claim 11 , wherein the mobile computing device performs one or more operations that enable the mobile computing device to interface with the second computing device.

20. The system of claim 11 , wherein the second computing device performs one or more operations to interface with the mobile computing device.

21. A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a mobile computing device, cause the one or more processors to:
in response to user inputs on a user interface generated on a display of the mobile computing device, transmit control information to a remote controlled vehicle to control movement of the remote controlled vehicle in a real-world environment;
interface with a second computing device that provides, via execution of a program by one or more processors of the second computing device, a virtual environment;
transmit position information, corresponding to the remote controlled vehicle to the second computing device to cause the second computing device to render a virtual entity upon the virtual environment, the virtual entity representing the remote controlled vehicle in the real-world environment;
receive, from the second computing device, data corresponding to the virtual entity within the virtual environment; and
display, on the display of the mobile computing device, the virtual environment including the virtual entity representing the remote controlled vehicle.

22. The non-transitory computer readable medium of claim 21 , wherein the executed instructions further cause the one or more processors to:
detect an event relating to the remote controlled vehicle in the real-world environment;
determine a virtual event based on the detected event; and
incorporate the virtual event into the virtual environ

Assignees

• Sphero Inc

Inventors

• Polo Fabrizio
• Carroll Jonathan
• Castator-Smith Skylar
• Ingram Ross

Classifications

• G06V10/255 Primary

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• A63F13/00 Primary

  Video games, i.e. games using an electronically generated display having two or more dimensions · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• A63F13/327

  using wireless networks, e.g. Wi-Fi® or piconet · CPC title

• A63F13/65

  automatically by game devices or servers from real world data, e.g. measurement in live racing competition · CPC title