Electronic device providing augmented reality and operating method thereof

Patent metadata

Field	Value
Publication number	US-12469229-B2
Application number	US-202318151683-A
Country	US
Kind code	B2
Filing date	Jan 9, 2023
Priority date	Jan 7, 2022
Publication date	Nov 11, 2025
Grant date	Nov 11, 2025

Abstract

An electronic device is provided. The electronic device includes a display, a camera, a memory, and at least one processor operatively connected to the display, the camera, and the memory, and based on an image obtained via the camera, the at least one processor may display, in the display, an augmented reality scene including at least one bookmarking object, may receive a user input for the augmented reality scene, and in response to the user input, store bookmarking data including at least one of information associated with the augmented reality scene or information associated with the at least one bookmarking object in the memory.

First claim

What is claimed is:

1 . An electronic device comprising:
a display; a camera; memory storing one or more computer programs; and one or more processors communicatively coupled to the display, the camera, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
display, in the display, an augmented reality scene comprising at least one bookmarking object based on an image obtained via the camera,
receive a first user input for capturing at least a portion of the augmented reality scene,
in response to capturing the at least a portion of the augmented reality scene based on the first user input, store, in the memory, bookmarking data comprising a subject image, information associated with the subject image, a virtual image, and information associated with the virtual image,
receive a second user input for searching for a captured augmented reality scene generated by capturing the at least a portion of the augmented reality scene, the second user input comprising a voice input,
extract at least one keyword from the voice input, and
search for bookmarking data comprising information corresponding to the at least one extracted keyword.

2 . The electronic device of claim 1 ,
wherein the at least one bookmarking object comprises at least one of a subject image recognized in the image obtained via the camera or a virtual image displayed by being included in the augmented reality scene, and wherein the virtual image comprises at least one of a virtual object or an application execution screen.

3 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
determine, based on the first user input, a capture area comprising the at least one bookmarking object from the augmented reality scene, and based on the determined capture area, determine the at least one bookmarking object included in the bookmarking data.

4 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
determine at least one of a location and a line of sight of a user of the electronic device, and determine the augmented reality scene based on at least one of the location and the line of sight.

5 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
segment the augmented reality scene into at least one area corresponding to the at least one bookmarking object, receive an additional user input for selecting an area among the at least one area; and in response to the additional user input, exclude a bookmarking object corresponding to the selected area from the at least one bookmarking object.

6 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
receive an additional user input for selecting information among information associated with the augmented reality scene or information associated with the at least one bookmarking object, and in response to the additional user input, exclude information selected from among the information associated with the augmented reality scene or the information associated with the at least one bookmarking object.

7 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to obtain, based on the information included in the bookmarking data, preference information associated with at least one of the bookmarking object or the information associated with the bookmarking object.

8 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
receive a third user input to information included in the bookmarking data, and control the display so as to display the bookmarking data in response to the third user input to the information included in the bookmarking data.

9 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
retrieve bookmarking data corresponding to the second user input for searching for the captured augmented reality scene in response to the second user input for searching for the captured augmented reality scene, and control the display so as to display an augmented reality scene corresponding to the retrieved bookmarking data.

10 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
receive a user command to transmit the bookmarking data, in response to the user command to transmit the bookmarking data, control the display so as to display a list of content sharing functions provided by the electronic device, and receive, as the user command, a third user input for selecting a sharing function that transmits the bookmarking data using at least one application in the list of content sharing functions.

11 . The electronic device of claim 1 , wherein the bookmarking data comprises basic information comprising type information that classifies the bookmarking object, type-based information configured based on the type information, augmented reality segment information indicating an area of the bookmarking object in the augmented reality scene, and an image obtained by capturing the augmented reality scene.

12 . The electronic device of claim 11 , wherein the basic information comprises at least one among location information, time information, or anchor information associated with the augmented reality scene.

13 . The electronic device of claim 11 ,
wherein, in a case that the type information indicates that the bookmarking object is a virtual object that overlaps the image, the type-based information comprises trigger information associated with a trigger that calls the virtual object or call information for calling the virtual object, and wherein, in a case that the type information is an application execution screen, the type-based information comprises state information of an application for executing the application execution screen.

14 . A method performed by an electronic device, the method comprising:
based on an image obtained via a camera, displaying, by the electronic device in a display, an augmented reality scene comprising at least one bookmarking object; receiving, by the electronic device, a first user input for capturing at least a portion of the augmented reality scene; in response to capturing the at least a portion of the augmented reality scene based on the first

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Jeoungju
• Kim Hyunwoo
• Park Chanmin
• Chae Sangwon
• Hwang Sunpil

Classifications

• G06F3/14

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title

• G10L15/08

  Speech classification or search · CPC title

• G06F16/9562

  Bookmark management · CPC title

• G10L2015/088

  Word spotting · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title