Augmentation of elements in data content

Patent metadata

Field	Value
Publication number	US-9292758-B2
Application number	US-201414148541-A
Country	US
Kind code	B2
Filing date	Jan 6, 2014
Priority date	May 14, 2012
Publication date	Mar 22, 2016
Grant date	Mar 22, 2016

Abstract

A system and method are disclosed for processing data content. Received data content comprises a scene and includes one or more recognized objects. The recognized object(s) include various characteristics, and can be detected and tracked during content capture. The data content can then be stored, and incorporated with metadata associated with the recognized object and/or one or more other elements in the data content. User inputs can be enabled in real-time, or post-capture, to augment one or more of the elements in the stored data content, including one or more characteristics of the recognized object. The data content can then be augmented to introduce one or more augmented elements, corresponding to respective elements of the data content, into the data content based on the user inputs.

First claim

What is claimed is:

1. A method of processing data content, the method being performed by one or more processors of a mobile computing device and comprising:
receiving video content using a capture component of the mobile computing device, the video content comprising a scene including a remotely operated self-propelled device;
detecting and tracking the self-propelled device over a duration of the video content;
generating metadata corresponding to the self-propelled device and each of one or more elements in the video content;
based on the generated metadata, displaying, on a display of the mobile computing device, a plurality of augmented reality elements selectable to augment each of the self-propelled device and the one or more elements;
receiving user selections from the plurality of augmented reality elements to augment the self-propelled device and the one or more elements of the video content; and
augmenting the self-propelled device and the one more elements in the video content by superimposing selected augmented reality elements over the self-propelled device and the one or more elements based on the user selections.

2. The method of claim 1 , further comprising:
receiving real-time data from the self-propelled device;
wherein the method is performed in real time while capturing the video content.

3. The method of claim 2 , further comprising:
outputting augmented video content, including the selected augmented reality elements, on the display of the mobile computing device in real time.

4. The method of claim 3 , wherein the one or more elements include an audio characteristic of the video content, and wherein the outputted augmented video content includes augmented audio content based on a user selection of an augmented reality element to augment the audio characteristic.

5. The method of claim 1 , further comprising:
storing the video content to enable a user of the mobile computing device to augment the stored video content after capture.

6. The method of claim 3 , wherein the real-time data received from the self-propelled device comprises one or more of orientation data, temperature data, or power level data of the self-propelled device, and wherein the selected augmented reality element for the self-propelled device visually represents the real-time data received from the self-propelled device.

7. The method of claim 6 , wherein the selected augmented reality element for the self-propelled device comprises one of a set of selectable virtual characters to represent the self-propelled device in the outputted augmented video content, and wherein the self-propelled device is remotely operated by a user via an augmented reality application executing on the mobile computing device.

8. A mobile computing device comprising:
a display;
one or more capturing components to capture data content;
one or more memory resources to store recorded data content, metadata associated with the recorded data content, and a plurality of augmented reality elements associated with the metadata; and
one or more processors to:
receive video content using the one or more capturing components, the video content comprising a scene including a remotely operated self-propelled device;
detect and track the self-propelled device over a duration of the video content;
generate metadata corresponding to the self-propelled device and each of one or more elements in the video content;
based on the generated metadata, display a set of the plurality of augmented reality elements selectable to augment each of the self-propelled device and the one or more elements of the video content;
receive user selections from the set of augmented reality elements to augment the self-propelled device and the one or more elements of the video content; and
augment the self-propelled device and the one or more elements in the video content by superimposing selected augmented reality elements over the self-propelled device and the one or more elements based on the user selections.

9. The mobile computing device of claim 8 , wherein the one or more processors are further to:
receive real-time data from the self-propelled device;
wherein the one or more processors augment the video content in real time while capturing the video content.

10. The mobile computing device of claim 9 , wherein the one or more processors are further to:
output augmented video content, including the selected augmented reality elements, on the display of the mobile computing device.

11. The mobile computing device of claim 10 , wherein the one or more elements include an audio characteristic of the video content, and wherein the outputted augmented video content includes augmented audio content based on a user selection of an augmented reality element to augment the audio characteristic.

12. The mobile computing device of claim 8 , wherein the one or more processors store the video content to enable a user of the mobile computing device to augment the stored video content after capture.

13. The mobile computing device of claim 10 , wherein the real-time data received from the self-propelled device comprises one or more of orientation data, temperature data, or power level data of the self-propelled device, and wherein the selected augmented reality element for the self-propelled device visually represents the real-time data from the self-propelled device.

14. The mobile computing device of claim 13 , wherein the selected augmented reality element for the self-propelled device comprises one of a set of selectable virtual characters to represent the self-propelled device in the outputted augmented video content, and wherein the self-propelled device is remotely operated by a user via an augmented reality application executing on the mobile computing device.

15. A non-transitory computer-readable medium, storing instructions that, when executed by one or more processors of a mobile computing device, cause the one or more processors to:
receive video content using a capture component of the mobile computing device, the video content comprising a scene including a remotely operated self-propelled device;
detect and track the self-propelled device over a duration of the video content;
generate metadata corresponding to the self-propelled device and each of one or more elements in the video content;
based on the generated metadata, display, on a display of the mobile computing device, a plurality of augmented reality elements selectable to augment each of the self-propelled device and the one or more elements;
receive user selections from the plurality of augmented reality elements to augment the self-propelled device and the one or more elements of the video content;
augment the self-propelled device and the one or more elements in the video content by superimposing selected augmented reality elements over the self-propelled device and the one or more elements based on the user selections.

16. The non-transitory computer-readable medium of claim 15 , wherein the instructions, when executed, further cause the one or more processors to:
receive real-time data from the self-propelled device; and
wherein the executed instructions cause the one or more processors to augment the video content in real time while capturing the video content.

17. The non-transitory computer-readable medium of claim 16 , wherein the instructions, when executed, further cause the one or more processors to:
output augmented video content, including the selected augmented reality elements, on the display of the mobile computing device in

Assignees

• Orbotix Inc
• Sphero Inc

Inventors

• Polo Fabrizio
• Castator-Smith Skylar
• Wilson Adam
• Smith Brian
• Martin Patrick
• Dorris Brandon

Classifications

• G06V10/255 Primary

  Detecting or recognising potential candidate objects based on visual cues, e.g. shapes · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06K9/3241 Primary

  Physics · mapped topic

• G06F3/041

  Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means · CPC title