Method for generating an augmented reality content and terminal using the same

Patent metadata

Field	Value
Publication number	US-10386918-B2
Application number	US-201414166321-A
Country	US
Kind code	B2
Filing date	Jan 28, 2014
Priority date	Jan 28, 2013
Publication date	Aug 20, 2019
Grant date	Aug 20, 2019

Abstract

A method for generating augmented reality content includes receiving an augmented reality mode execution request for generating augmented reality content based on electronic book content, activating a camera in response to the request to provide a preview image, loading book data of the electronic book on the preview image, detecting an event occurring in the augmented reality mode, retrieving reaction data pre-mapped to the event, and loading the reaction data on the preview image.

First claim

What is claimed is:

1. An augmented reality content generation method in a mobile terminal having a touch screen, a camera and a sensor unit, comprising:
receiving, while displaying content, an augmented reality mode execution request for generating the augmented reality content based on the content;
executing the camera in response to the request, and displaying, on the touch screen, a preview image captured by the camera, the preview image being a live image of a real environment of a user;
loading default book data of the electronic book on the preview image;
storing, in a mapping table, reaction data mapped to motions of the user and sensor values generated by the sensor unit;
detecting an event occurring in the augmented reality mode, wherein detecting the event comprises at least one of detecting a motion of the user in the preview image and detecting the sensor values obtained by the sensor unit;
extracting the reaction data pre-mapped to the event;
loading the extracted reaction data on the preview image;
receiving an augmented reality mode termination request; and
in response to the augmented reality mode termination request, ending the augmented reality mode, and storing the default book data, the reaction data and the preview image,
wherein the default book data and the reaction data are loaded on the preview image.

2. The method of claim 1 , further comprising selecting the augmented reality content to be presented on the preview image from the content in the augmented reality mode.

3. The method of claim 1 , further comprising:
recognizing an object, person, and background of a real environment provided as the preview image; and
determining the coordinates of the object, person, and background on the preview image.

4. The method of claim 3 , wherein the reaction data is output at an identical coordinate area as the augmented reality content or another coordinate area adjacent to the augmented reality content.

5. The method of claim 3 , wherein the augmented reality content is output on an area of reference coordinates recognized as a flat surface through coordinate detection on the preview image.

6. The method of claim 1 , wherein the sensor unit comprises at least one of acceleration sensor, gravity sensor, compass sensor, gyro sensor, temperature sensor, and motion sensor.

7. The method of claim 1 , wherein the reaction data changes a shape of the augmented reality content or replaces the augmented reality content according to the detection of the event.

8. The method of claim 1 , wherein the augmented reality content comprises at least one of character image data, audio data, vibration data, video data, 2-Dimensional (2D) graphic data, and 3D graphic data, with the exception of basic background image data.

9. The method of claim 1 , further comprising:
outputting the augmented reality content on the preview image,
wherein outputting the augmented reality content comprises:
converting text data of the content to speech data; and
outputting the speech data.

10. A mobile terminal configured to generate augmented reality content, the terminal comprising:
a touchscreen;
a camera;
a speaker;
a sensor unit
a memory; and
a processor,
wherein the processor is configured to:
receive, while displaying content, an augmented reality mode execution request for generating the augmented reality content based on the content,
execute the camera in response to the request, and display, on the touch screen, a preview image captured by the camera, the preview image being a live image of a real environment of a user,
load default book data of the electronic book on the preview image,
store, in a mapping table, reaction data mapped to motions of the user and sensor values generated by the sensor unit,
detect an event occurring in the augmented reality mode, wherein detecting the event comprises at least one of detecting a motion of the user in the preview image and detecting the sensor values obtained by the sensor unit,
extract the reaction data pre-mapped to the event,
load the extracted reaction data on the preview image,
receive an augmented reality mode termination request, and
in response to the augmented reality mode termination request, end the augmented reality mode, and store the default book data, the reaction data and the preview image,
wherein the default book data and the reaction data are stored on the preview image.

11. The terminal of claim 10 , wherein the processor comprises:
an electronic book data acquisition module which parses at least one page of the electronic book to extract the book data and sends the book data to the display unit;
a preview image acquirer which acquires the preview image from the camera;
a default data extractor which extracts the augmented reality content from the content to be output onto the preview image in the augmented reality mode;
a specific event detector which detects motion of the user or acquires sensor values in the augmented reality mode;
a reaction data extractor which extracts the reaction data corresponding to the motion of the user or the sensor values; and
a manager module which activates the preview image acquirer, default data extractor, specific event detector, and reaction data extractor in response to the augmented reality mode execution request.

12. The terminal of claim 10 , wherein the processor is further configured to recognize an object, person, and background of a real environment provided as the preview image and to determine the coordinates of the object, person, and background on the preview image.

13. The terminal of claim 12 , wherein the reaction data is output at an identical coordinate area as the augmented reality content or other coordinate area adjacent to the augmented reality content.

14. The terminal of claim 12 , wherein the augmented reality content is output on an area of reference coordinates recognized as a flat surface through coordinate detection on the preview image.

15. The terminal of claim 10 , wherein the reaction data changes a shape of the augmented reality content or replaces the augmented reality content according to the detection of the event.

16. The terminal of claim 10 , wherein the augmented reality content comprises at least one of character image data, audio data, vibration data, video data, 2-Dimensional (2D) graphic data, and 3D graphic data, with the exception of basic background image data.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Shin Daekyu
• Cho Kyusung
• Cho Ikhwan

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• H04B1/40

  Circuits · CPC title

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title