Electronic device and operation method thereof

Patent metadata

Field	Value
Publication number	US-2024411359-A1
Application number	US-202418809907-A
Country	US
Kind code	A1
Filing date	Aug 20, 2024
Priority date	Feb 21, 2022
Publication date	Dec 12, 2024
Grant date	—

Abstract

An electronic device that is an augmented reality (AR) device is provided. The electronic device includes sensor circuitry configured to recognize a user's finger. memory storing one or more computer programs, and one or more processors communicatively coupled to the sensor circuitry and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to, after a result of recognizing the user's finger is received from the sensor circuitry, determine whether there is accuracy correction data of coordinates of the user's finger, when there is the accuracy correction data of the coordinates of the user's finger, correct an accuracy of the coordinates of the user's finger, when there is no accuracy correction data of the coordinates of the user's finger, recognize the user's finger in an image obtained by the sensor circuitry and calculate the coordinates of the user's finger in the AR environment based on data of the user's finger, and correct the coordinates of the user's finger in the AR environment.

First claim

What is claimed is:

1 . An electronic device that is an augmented reality (AR) device, the electronic device comprising:
sensor circuitry configured to recognize a user's finger; memory storing one or more computer programs; and one or more processors communicatively coupled to the sensor circuitry and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
after a result of recognizing the user's finger is received from the sensor circuitry, determine whether there is accuracy correction data of coordinates of the user's finger,
when there is the accuracy correction data of the coordinates of the user's finger, correct an accuracy of the coordinates of the user's finger,
when there is no accuracy correction data of the coordinates of the user's finger, recognize the user's finger in an image obtained by the sensor circuitry and calculate the coordinates of the user's finger in an AR environment based on data of the user's finger, and
correct the coordinates of the user's finger in the AR environment.

2 . The electronic device of claim 1 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
when the accuracy of the coordinates of the user's finger in the AR environment needs to be corrected, determine whether a user wants to correct the accuracy of the coordinates of the user's finger in the AR environment, and when the user wants to correct the accuracy of the coordinates of the user's finger in the AR environment, perform an accuracy measurement of the coordinates of the user's finger.

3 . The electronic device of claim 2 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
display a selection menu through a user interface for identifying whether the user wants to correct the accuracy of the coordinates of the user's finger in the AR environment, and perform the accuracy measurement of the coordinates of the user's finger in the AR environment based on a result of a user's selection.

4 . The electronic device of claim 3 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
display, in a planar space, an object for user recognition that includes a picture of a touch position representing a position touched by the user's finger together with a plurality of markers for correcting the coordinates of the user's finger.

5 . The electronic device of claim 4 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
display the picture of the touch position and the plurality of markers on a screen of a display of an external electronic device.

6 . The electronic device of claim 5 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
display the picture of the touch position in a center of the screen of the display, and display two markers on left and right sides of the screen of the display, or display four markers at four corners of the screen of the display.

7 . The electronic device of claim 4 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
calculate first coordinates touched by the user's finger based on a position of the picture of the touch position, and calculate second coordinates of a virtual finger in the AR environment based on the plurality of markers and the data of the user's finger touching the picture at the touch position.

8 . The electronic device of claim 7 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
perform an accuracy measurement of the coordinates of the user's finger in the AR environment based on a difference between the first coordinates touched by the user's finger and the second coordinates touched by the virtual finger in the AR environment, and terminate the accuracy measurement when the difference between the first coordinates and the second coordinates is within a preset distance.

9 . The electronic device of claim 7 , wherein the one or more computer programs further include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to:
when the correction of the coordinates of the user's finger in the AR environment is required, calculate a difference between the first coordinates touched by the user's finger and the second coordinates touched by the virtual finger in the AR environment, and correct the coordinates of the user's finger in the AR environment based on the difference between the first coordinates and the second coordinates.

10 . The electronic device of claim 1 , wherein the sensor circuitry includes at least one of a camera or at least one radar.

11 . An operating method performed by an electronic device that provides a user with an augmented reality (AR) environment, the operating method comprising:
recognizing, by the electronic device via sensor circuitry of the electronic device, a user's finger; after receiving a result of recognizing the user's finger, determining, by the electronic device, whether there is accuracy correction data of coordinates of the user's finger; correcting, by the electronic device, the accuracy of the coordinates of the user's finger based on determining that there is the accuracy correction data of the coordinates of the user's finger; recognizing, by the electronic device, the user's finger in an image obtained by the sensor circuitry based on determining that there is no accuracy correction data of the coordinates of the user's finger and calculating, by the electronic device, the coordinates of the user's finger in the AR environment based on data of the user's finger; and correcting, by the electronic device, the coordinates of the user's finger in the AR environment.

12 . The operating method of claim 11 , further comprising:
based on the accuracy of the coordinates of the user's finger in the AR environment needing to be corrected, determining, by the electronic device, whether the user wants to correct the accuracy of the coordinates of the user's finger in the AR environment; and based on determining that the user wants to correct the accuracy of the coordinates of the user's finger in the AR environment, performing, by the electronic device, an accuracy measurement of the coordinates of the user's finger.

13 . The operating method of claim 12 , further comprising:
displaying, by the electronic device, a selection menu through a user interface for identifying whether the user wants to correct the accuracy of the coordinates of the user's finger in the AR environment; and performing, by the electronic device, the accuracy measurement of

Assignees

• Samsung Electronics Co Ltd

Inventors

• Woo Hyuntaek
• Kang Jinmo
• Kim Beomsu
• Lee Gunhee
• Choi Daiwoong

Classifications

• G06V40/107

  Static hand or arm · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06F3/04886

  by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title