Controlling a user interface with a trackpad and a smart watch

Patent metadata

Field	Value
Publication number	US-2024036646-A1
Application number	US-202318119715-A
Country	US
Kind code	A1
Filing date	Mar 9, 2023
Priority date	Jul 28, 2022
Publication date	Feb 1, 2024
Grant date	—

Abstract

Methods and apparatuses for controlling a user interface with a wearable device, such as a smart watch, are disclosed. In some embodiments, the method includes detecting, by an extend reality (XR) device, a wearable device, communicably connecting the XR device to the wearable device, and manipulating a user interface in a virtual environment displayed on a display in the XR device with the wearable device.

First claim

We claim:

1 . A method comprising:
detecting, by an extend reality (XR) device, a wearable device; communicably connecting the XR device to the wearable device; and manipulating a user interface in a virtual environment displayed on a display in the XR device with the wearable device.

2 . The method of claim 1 wherein manipulating the user interface comprises moving a cursor with the user interface based on detected motion made by a user on the wearable device that is sent to the XR device, the detected motion being moving one or more finger of the user across a display surface of the wearable device.

3 . The method of claim 1 wherein manipulating the user interface comprises selecting an object being displayed on the user interface based on detected motion made by a user on the wearable device that is sent to the XR device, the detected motion being a tap by the user on the display surface of the wearable device.

4 . The method of claim 1 wherein manipulating the user interface comprises scrolling a window being displayed on the user interface based on detected motion made by a user on the wearable device that is sent to the XR device, the detected motion being movement of one or more fingers by the user across a portion of the wearable device.

5 . The method of claim 4 wherein the at least one finger comprises two fingers and wherein the portion of the wearable device comprises a display of the wearable device.

6 . The method of claim 1 further comprising displaying a virtual interface in the user interface and in proximity to the wearable device and including user-selectable graphical user interface (GUI) elements.

7 . The method of claim 6 wherein the user-selectable GUI elements are virtual buttons.

8 . The method of claim 6 further comprising:
capturing movements of one or more fingers made by a user in relation to one of the GUI elements;
recognizing the movements as a selection of the one GUI element; and
invoking a feature associated with the one GUI element in response to recognizing the movements.

9 . The method of claim 1 wherein the virtual GUI elements are customizable to associate different features with each of the virtual GUI elements.

10 . The method of claim 1 further comprising:
identifying, using a camera, a marker on the wearable device; and
detecting position of the wearable device using the marker,
wherein displaying the virtual interface in the user interface in proximity to the wearable device comprises displaying the virtual interface based on the position of the wearable device.

11 . The method of claim 10 wherein detecting position of the wearable device using the marker comprises detecting position of a display of the wearable device using the marker.

12 . The method of claim 1 wherein the virtual interface is displayed around the wearable device, and the wearable device comprises a smart watch device.

13 . A computing system for controlling a user interface, the computing system comprising:
one or more processors; and one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
detecting, by an extend reality (XR) device, a wearable device;
communicably connecting the XR device to the wearable device; and
manipulating a user interface in a virtual environment displayed on a display in the XR device with the wearable device.

14 . The computing system of claim 13 wherein manipulating the user interface comprises moving a cursor with the user interface based on detected motion made by a user on the wearable device that is sent to the XR device, the detected motion being moving one or more finger of the user across a display surface of the wearable device.

15 . The computing system of claim 13 wherein manipulating the user interface comprises selecting an object being displayed on the user interface based on detected motion made by a user on the wearable device that is sent to the XR device, the detected motion being a tap by the user on the display surface of the wearable device.

16 . The computing system of claim 13 wherein manipulating the user interface comprises scrolling a window being displayed on the user interface based on detected motion made by a user on the wearable device that is sent to the XR device, the detected motion being movement of one or more fingers by the user across a portion of the wearable device.

17 . The computing system of claim 13 wherein the process further comprises: displaying a virtual interface in the user interface and in proximity to the wearable device and including user-selectable graphical user interface (GUI) elements.

18 . The computing system of claim 17 wherein the user-selectable GUI elements are virtual buttons.

19 . The computing system of claim 17 wherein the process further comprises:
capturing movements of one or more fingers made by a user in relation to one of the GUI elements;
recognizing the movements as a selection of the one GUI element; and
invoking a feature associated with the one GUI element in response to recognizing the movements.

20 . One or more non-transitory computer readable storage media having instructions stored thereupon which, when executed by a system having at least a processor and a memory therein, cause the system to perform operations comprising:
detecting, by an extend reality (XR) device, a wearable device; communicably connecting the XR device to the wearable device; and manipulating a user interface in a virtual environment displayed on a display in the XR device with the wearable device.

Assignees

• Ntt Docomo Inc

Inventors

• Yu Young-Cheol
• Tanaka Yuya
• Osako Yoji

Classifications

• G06F3/038

  Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry · CPC title

• G06F3/014 Primary

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G06F3/0488

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/0485

  Scrolling or panning · CPC title

• G06F2203/04104

  Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger · CPC title