Electronic device for processing user input and method thereof

Patent metadata

Field	Value
Publication number	US-2022179497-A1
Application number	US-202117457976-A
Country	US
Kind code	A1
Filing date	Dec 7, 2021
Priority date	Dec 7, 2020
Publication date	Jun 9, 2022
Grant date	—

Abstract

An electronic device according to an example embodiment of the disclosure may include an at least partially transparent display, a camera, a memory, and a processor operatively connected with the at least partially transparent display, the camera, and the memory. The memory may store instructions which, when executed, may cause the processor to receive a first gesture input using a part of a human body of a user using the camera, receive an utterance input of the user in response to the first gesture input, display text generated based on a property of the utterance input on coordinates corresponding to a position of a part of the human body of the user on augmented reality (AR) provided via the at least partially transparent display, end the reception of the utterance input, when receiving an end gesture input using a part of the human body of the user using the camera, and fix the displayed text in a specified position on the augmented reality (AR), in response to the received end gesture input. In addition, various embodiments recognized through the specification are possible.

First claim

1 . An electronic device, comprising:
an at least partially transparent display; a camera; a memory; and at least one processor operatively connected with the at least partially transparent display, the camera, and the memory, wherein the memory stores instructions that, when executed, are configured to cause the at least one processor to: receive a first gesture input via at least a part of a human body of a user via the camera; receive an utterance input of the user in response to the first gesture input; display text generated based on at least a property of the utterance input at coordinates corresponding to a position of at least a part of the human body of the user on augmented reality (AR) via the at least partially transparent display; end the reception of the utterance input, when receiving an end gesture input via at least a part of the human body of the user via the camera; and fix the displayed text in a specified position on the augmented reality (AR), in response to at least the received end gesture input.

2 . The electronic device of claim 1 , wherein the end gesture is a gesture having a similarity of a threshold or more with the first gesture.

3 . The electronic device of claim 1 , wherein the processor is configured to receive a second gesture input having a similarity of a threshold or more with the first gesture using the camera, edit the generated text depending on a function corresponding to the second gesture input, and display the edited text on the at least partially transparent display.

4 . The electronic device of claim 3 , wherein the function corresponding to the second gesture input comprises adjusting at least one of a color, a size, a shape, or a slope of the generated text.

5 . The electronic device of claim 3 , wherein the function corresponding to the second gesture input comprises setting an open range of the generated text.

6 . The electronic device of claim 3 , wherein the function corresponding to the second gesture input comprises mapping to perform a predetermined operation, when the processor is configured to receive an input using the text fixed in the specified position on the AR.

7 . The electronic device of claim 3 , wherein the processor is configured to select one of a plurality of selectable options of the function corresponding to the second gesture input based on at least a number of times of inputting the second gesture input.

8 . The electronic device of claim 3 , wherein the processor is configured to select one of a plurality of selectable options based on at least a similarity between the second gesture and the first gesture.

9 . The electronic device of claim 1 , wherein the processor is configured to determine coordinates to place the displayed text on the AR based on at least a change in position of the end gesture input, when receiving the end gesture input, and fix the displayed text in a position corresponding to the determined coordinates.

10 . The electronic device of claim 1 , wherein the fix of the text comprises placement of the text such that the displayed text is displayed at a specified position on the AR.

11 . A method performed by an electronic device, the method comprising:
receiving a first gesture input via a part of a human body of a user via a camera included in the electronic device; receiving an utterance input of the user in response to at least the first gesture input; displaying, via an at least partially transparent display of the electronic device, text generated based on a property of the utterance input at coordinates corresponding to a position of a part of the human body of the user on augmented reality (AR); ending reception of the utterance input, based on at least receiving an end gesture input via a part of the human body of the user via the camera; and providing the displayed text in a specified position on the AR, in response to at least the received end gesture input.

12 . The method of claim 11 , wherein the end gesture comprises a gesture having a similarity of a threshold or more with the first gesture.

13 . The method of claim 11 , further comprising:
receiving a second gesture input having a similarity of a threshold or more with the first gesture via a sensor of the electronic device; editing the generated text depending on at least a function corresponding to the second gesture input; and displaying the edited text via the at least partially transparent display.

14 . The method of claim 13 , wherein the function corresponding to the second gesture input comprises adjusting at least one of a color, a size, a shape, or a slope of the generated text.

15 . The method of claim 13 , wherein the function corresponding to the second gesture input comprises setting an open range of the generated text.

16 . The method of claim 13 , wherein the function corresponding to the second gesture input comprises mapping to perform a predetermined operation, when the electronic device receives an input using the text fixed in the specified position on the AR.

17 . The method of claim 13 , further comprising:
selecting one of selectable options of the function corresponding to the second gesture input based on at least a number of times of inputting the second gesture input.

18 . The method of claim 13 , further comprising:
selecting one of selectable options using at least one of the number of the selectable options and a similarity between the second gesture and the first gesture.

19 . The method of claim 11 , further comprising:
determining coordinates to place the displayed text on the AR based on at least a change in position of the end gesture input, when receiving the end gesture input; and fixing the displayed text in a position corresponding to a position of the determined coordinates.

20 . The method of claim 11 , wherein providing the text may be placing the text such that the displayed text is displayed at a specified position on the AR.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Jung Shinjae
• Myung Insik
• Lee Seungjoon
• Lee Joayoung
• Choi Inyoung

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/0482

  Interaction with lists of selectable items, e.g. menus · CPC title

• G06F2203/0381

  Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer · CPC title

• G06F3/04847

  Interaction techniques to control parameter settings, e.g. interaction with sliders or dials · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title