Methods, devices, and systems for displaying a user interface on a user and detecting touch gestures

Patent metadata

Field	Value
Publication number	US-11042221-B2
Application number	US-202017037488-A
Country	US
Kind code	B2
Filing date	Sep 29, 2020
Priority date	Jan 8, 2018
Publication date	Jun 22, 2021
Grant date	Jun 22, 2021

Abstract

An example method of identifying a touch gesture on a user is provided. The method includes receiving, by one or more transducers of a wearable device attached to an appendage of the user, a set of signals that propagate through the user's appendage and establish a signal pathway to the wearable device. The method also includes, while receiving the set of signals, determining baseline characteristics for the signal pathway, and sensing a change in the baseline characteristics caused by user interaction with an affordance of a user interface projected or perceived on the user's appendage. The method further includes, in accordance with a determination that the sensed change in the baseline characteristics satisfies a contact criterion, reporting a candidate touch event on the user's appendage to a separate electronic device that creates the user interface or is in communication with another electronic device that creates the user interface.

First claim

What is claimed is:

1. A method, comprising:
at a wearable device that (i) includes one or more transducers and (ii) is configured to attach to an appendage of a user:
receiving, by the one or more transducers, a set of signals that establish a signal pathway to the wearable device, wherein the signals in the set of signals propagate through the user's appendage;
while receiving the set of signals:
determining baseline characteristics for the signal pathway; and
sensing a change in the baseline characteristics caused by user interaction with an affordance of a user interface projected or perceived on the user's appendage; and

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion, reporting a candidate touch event on the user's appendage to a separate electronic device, wherein the separate electronic device creates the user interface or is in communication with another electronic device that creates the user interface.

2. The method of claim 1 , wherein reporting the candidate touch event comprises sending transducer data corresponding to the sensed change in the baseline characteristics to the separate electronic device.

3. The method of claim 2 , further comprising, at the wearable device:
determining an approximate location of the candidate touch event on the user's appendage based, at least in part, on the sensed change in the baseline characteristics,
wherein the transducer data sent to the separate electronic device further comprises information indicating the approximate location of the candidate touch event.

4. The method of claim 2 , wherein the transducer data sent to the separate electronic device also indicates an approximate location of the candidate touch event on the user's appendage.

5. The method of claim 2 , further comprising:
capturing, via one or more cameras of the separate electronic device, the candidate touch event;
generating, by the separate electronic device, image data according to the capturing of the candidate touch event; and
executing, by the separate electronic device, a function associated with the affordance of the user interface after processing the transducer data and the image data.

6. The method of claim 1 , further comprising:
displaying, by the separate electronic device, the affordance of the user interface on the user's appendage.

7. The method of claim 1 , wherein:
the baseline characteristics include a baseline phase value; and
sensing the change in the baseline characteristics for the signal pathway comprises detecting a phase value of the signal pathway that differs from the baseline phase value.

8. The method of claim 7 , wherein:
the contact criterion includes a phase difference threshold; and
reporting the candidate touch event is performed in accordance with a determination that a difference between the phase value and the baseline phase value satisfies the phase difference threshold.

9. The method of claim 1 , wherein:
the baseline characteristics include a baseline amplitude value; and
sensing the change in the baseline characteristics for the signal pathway comprises detecting an amplitude value of the signal pathway that differs from the baseline amplitude value.

10. The method of claim 9 , wherein:
the contact criterion includes an amplitude difference threshold; and
reporting the candidate touch event is performed in accordance with a determination that a difference between the amplitude value and the baseline amplitude value satisfies the amplitude difference threshold.

11. The method of claim 1 , wherein:
the baseline characteristics include a baseline amplitude value and a baseline phase value; and
sensing the change in the baseline characteristics for the signal pathway comprises detecting (i) an amplitude value of the signal pathway that differs from the baseline amplitude value, and (ii) a phase value of the signal pathway that differs from the baseline phase value.

12. The method of claim 11 , wherein:
the contact criterion includes an amplitude difference threshold and a phase difference threshold; and
reporting the candidate touch event is performed in accordance with a determination that: (i) a difference between the amplitude value and the baseline amplitude value satisfies the amplitude difference threshold, and (ii) a difference between the phase value and the baseline phase value satisfies the phase difference threshold.

13. The method of claim 1 , wherein:
the contact criterion includes a time threshold;
sensing the change in the baseline characteristics comprises sensing the change for a period of time; and
reporting the candidate touch event is performed in accordance with a determination that the period of time satisfies the time threshold.

14. The method of claim 1 , further comprising, at the wearable device:
before receiving the set of signals:
receiving a plurality predetermined values for signals characteristics, wherein each of the predetermined values for the signals characteristics corresponds to a specific location of the appendage of the user.

15. The method of claim 1 , wherein the candidate touch event is selected from the group consisting of: a tap gesture, a press-and-hold gesture, a multi-tap gesture, a swipe gesture, a drag gesture, a pinch gesture, a pull gesture, a hover, and a twist gesture.

16. The method of claim 1 , wherein:
reporting the candidate touch event comprises sending, to the separate electronic device, data associated with the sensed change in the signal pathway; and
the separate electronic device determines whether the user intended to interact with the affordance of the user interface displayed on the user's appendage based, at least in part, on the data associated with the sensed change in the signal pathway.

17. The method of claim 1 , wherein the separate electronic device is an artificial-reality system selected from the group consisting of: an augmented-reality system, a virtual-reality system, and a mixed-reality system.

18. A wearable device attached to an appendage of a user, the wearable device comprising:
one or more transducers;
one or more processors; and
memory storing one or more programs, which when executed by the one or more processors cause the wearable device to:
receive, by the one or more transducers, a set of signals that establish a signal pathway to the wearable device, wherein the signals in the set of signals propagate through the user's appendage;
while receiving the set of signals:
determine baseline characteristics for the signal pathway; and
sense a change in the baseline characteristics caused by user interaction with an affordance of a user interface projected or perceived on the user's appendage; and

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion, report a candidate touch event on the user's appendage to a separate electronic device, wherein the separate electronic device creates the user interface or is in communication with another electronic device that creates the user interface.

19. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of a wearable device having one or more transducers and configured to attach to an appendage of a user, the one or more programs including instructions, which when executed by the one or more processors, cause the wearable device to:
re

Assignees

• Facebook Tech Llc

Inventors

• Keller Sean Jason
• Trutna Tristan Thomas
• Benko Hrvoje

Classifications

• G06F1/1673

  Arrangements for projecting a virtual keyboard · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F1/163

  Wearable computers, e.g. on a belt · CPC title