Wearable device for recognizing touch input by external object and method thereof

Patent metadata

Field	Value
Publication number	US-12554375-B2
Application number	US-202418428454-A
Country	US
Kind code	B2
Filing date	Jan 31, 2024
Priority date	Mar 28, 2023
Publication date	Feb 17, 2026
Grant date	Feb 17, 2026

Abstract

A wearable device may obtain an image, while a screen is displayed, using a camera. The wearable device may identify a visual object corresponding to an external object from the image. The wearable device may display, at a first location of a portion of the visual object corresponding to a portion of the external object farther than a reference distance from a second surface, a pointer on the screen. The wearable device may recognize the external object contacted on the second surface according to movement from outside the reference distance, as a touch input at a second location identified based on a path of the movement and the first location, and provide feedback with respect to the touch input.

First claim

What is claimed is:

1 . A wearable device, comprising:
a housing; memory storing one or more programs; one or more processors; a display positioned on a first surface of the housing; and a camera positioned toward a direction different from a direction of the display, wherein the one or more programs include instructions that, when executed by the one or more processors, cause the wearable device to:
obtain, while a screen is displayed on the display, an image using the camera,
identify a visual object corresponding to an external object from the image,
display, at a first location of a portion of the visual object corresponding to a portion of the external object farther than a reference distance from a second surface opposite to the first surface, a pointer on the screen, and
recognize the external object contacted on the second surface according to movement of the external object from outside the reference distance, as a touch input at a second location identified based on a path of the movement and the first location, and provide feedback with respect to the touch input.

2 . The wearable device of claim 1 , further comprising:
one or more sensors for identifying the external object on the second surface, wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
identify the external object inside of the reference distance from the second surface by using the one or more sensors, and
identify the second location based on a third location of the portion of the external object within the reference distance.

3 . The wearable device of claim 2 , wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
adjust a transparency of the screen based on a distance from the second surface to the external object and maintain brightness of the pointer.

4 . The wearable device of claim 2 , wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
identify the first location by using the camera within a preset distance from the reference distance; identify the third location by using the one or more sensors; and identify the second location based on the third location and the first location.

5 . The wearable device of claim 2 , wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
switch between using the camera and using the one or more sensors to identify distance of the external object from the second surface based on the external object being a preset distance from the second surface.

6 . The wearable device of claim 1 , wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
move, based on displaying the pointer, a virtual object providing the feedback with respect to the touch input, to a distance able to be selected by the pointer.

7 . The wearable device of claim 1 , wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
adjust the reference distance based on brightness of the visual object.

8 . The wearable device of claim 1 , wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
initiate, based on identifying the visual object corresponding to a preset third surface of the external object, displaying of the pointer in the image.

9 . The wearable device of claim 8 , wherein the one or more programs further include instructions that, when executed by the one or more processors, cause the wearable device to:
based on identifying the visual object corresponding to a fourth surface different from the third surface of the external object, cease to display the pointer.

10 . A method performed by a wearable device, the method comprising:
obtaining, while a screen is displayed on a display of the wearable device positioned on a first surface of a housing of the wearable device, an image using a camera of the wearable device positioned toward a direction different from a direction of the display; identifying a visual object corresponding to an external object from the image; displaying, at a first location of a portion of the visual object corresponding to a portion of the external object farther than a reference distance from a second surface opposite to the first surface, a pointer on the screen; and recognizing the external object contacted on the second surface according to movement of the external object from outside the reference distance, as a touch input at a second location identified based on a path of the movement and the first location, and providing feedback with respect to the touch input.

11 . The method of claim 10 , further comprising:
identifying the external object inside of the reference distance from the second surface by using one or more sensors of the wearable device for identifying the external object on the second surface; and identifying the second location based on a third location of the portion of the external object within the reference distance.

12 . The method of claim 11 , wherein the displaying of the pointer comprises:
adjusting a transparency of the screen based on a distance from the second surface to the external object and maintain brightness of the pointer.

13 . The method of claim 11 , wherein the identifying of the second location comprises:
identifying the first location by using the camera within a preset distance from the reference distance; identifying the third location by using the one or more sensors; and identifying the second location based on the third location and the first location.

14 . The method of claim 11 , wherein the providing comprises:
moving, based on displaying the pointer, a virtual object providing the feedback with respect to the touch input, to a distance able to be selected by the pointer.

15 . The method of claim 11 , further comprising:
switching between using the camera and using the one or more sensors to identify distance of the external object from the second surface based on the external object being a preset distance from the second surface.

16 . The method of claim 10 , further comprising:
adjusting the reference distance based on brightness of the visual object.

17 . The method of claim 10 , wherein the displaying of the pointer comprises:
initiating, based on identifying the visual object corresponding to a preset third surface of the external object, display of the pointer in the image.

18 . The method of claim 17 , further comprising:
based on identifying the visual object corresponding to a fourth surface different from the third surface of the external object, ceasing to display the pointer.

19 . One or more non-transitory computer readable storage media storing one or more programs including instructions that, when executed by one or more processors of a wearable device, cause the wearable device to perform operations, the operations comprising:
obtaining, while a screen is displayed on a display of the wearable device positioned on a first surface of a housing of the wearable device, an image using a camera of the wearable device positioned toward a direction diff

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Sanghun
• Yeom Donghyun
• Choi Sungsoo
• Kim Sungoh
• Kim Jihyun
• Ahn Sunggeun
• Lee Gunhee

Classifications

• G02B27/017

  Head mounted · CPC title

• G06V40/28 Primary

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/0481

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title