Methods, devices, and systems for displaying a user interface on a user and detecting touch gestures

Patent metadata

Field	Value
Publication number	US-11467670-B2
Application number	US-202117354911-A
Country	US
Kind code	B2
Filing date	Jun 22, 2021
Priority date	Mar 23, 2018
Publication date	Oct 11, 2022
Grant date	Oct 11, 2022

Abstract

An example method of identifying a touch gesture on a user is provided. The method includes receiving, by one or more transducers of a wearable device, a set of signals that establish a signal pathway to the wearable device. The method also includes, while receiving the set of signals, determining baseline characteristics for the signal pathway, and sensing a change in the baseline characteristics caused by user interaction with an affordance of a user interface projected or perceived on the user's appendage. The method further includes, in accordance with a determination that the sensed change in the baseline characteristics satisfies a contact criterion, reporting a candidate touch event on the user's appendage to a separate electronic device that creates the user interface or is in communication with another electronic device that creates the user interface.

First claim

What is claimed is:

1. A method, comprising:
at a wearable device that includes one or more transducers:
receiving, by the one or more transducers, a set of signals that establish a signal pathway to the wearable device;
while receiving the set of signals:
determining baseline characteristics for the signal pathway; and
sensing, in the set of signals, a change from baseline characteristics for the signal pathway caused by user interaction with a projected affordance of a user interface; and

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion, reporting a candidate touch event to a separate electronic device, wherein the separate electronic device creates the user interface or is in communication with another electronic device that creates the user interface.

2. The method of claim 1 , wherein reporting the candidate touch event comprises sending transducer data corresponding to the sensed change in the baseline characteristics to the separate electronic device.

3. The method of claim 2 , further comprising, at the wearable device:
determining an approximate location of the candidate touch event based, at least in part, on the sensed change in the baseline characteristics,
wherein the transducer data sent to the separate electronic device further comprises information indicating the approximate location of the candidate touch event.

4. The method of claim 2 , wherein the transducer data sent to the separate electronic device also indicates an approximate location of the candidate touch event.

5. The method of claim 2 , further comprising:
capturing, via one or more cameras of the separate electronic device, the candidate touch event;
generating, by the separate electronic device, image data according to the capturing of the candidate touch event; and
executing, by the separate electronic device, a function associated with the affordance of the user interface after processing the transducer data and the image data.

6. The method of claim 1 , further comprising:
displaying, by the separate electronic device, the affordance of the user interface on a user's appendage.

7. The method of claim 6 , further comprising, at the wearable device:
before receiving the set of signals:
receiving a plurality of predetermined values for signal characteristics, wherein each of the predetermined values for the signal characteristics corresponds to a specific location of the appendage of the user.

8. The method of claim 6 , wherein:
reporting the candidate touch event comprises sending, to the separate electronic device, data associated with the sensed change in the signal pathway; and
the separate electronic device determines whether the user intended to interact with the affordance of the user interface displayed on the user's appendage based, at least in part, on the data associated with the sensed change in the signal pathway.

9. The method of claim 1 , wherein:
the baseline characteristics include a baseline phase value; and
sensing the change in the baseline characteristics for the signal pathway comprises detecting a phase value of the signal pathway that differs from the baseline phase value.

10. The method of claim 9 , wherein:
the contact criterion includes a phase difference threshold; and
reporting the candidate touch event is performed in accordance with a determination that a difference between the phase value and the baseline phase value satisfies the phase difference threshold.

11. The method of claim 1 , wherein:
the baseline characteristics include a baseline amplitude value; and
sensing the change in the baseline characteristics for the signal pathway comprises detecting an amplitude value of the signal pathway that differs from the baseline amplitude value.

12. The method of claim 11 , wherein:
the contact criterion includes an amplitude difference threshold; and
reporting the candidate touch event is performed in accordance with a determination that a difference between the amplitude value and the baseline amplitude value satisfies the amplitude difference threshold.

13. The method of claim 1 , wherein:
the baseline characteristics include a baseline amplitude value and a baseline phase value; and
sensing the change in the baseline characteristics for the signal pathway comprises detecting (i) an amplitude value of the signal pathway that differs from the baseline amplitude value, and (ii) a phase value of the signal pathway that differs from the baseline phase value.

14. The method of claim 13 , wherein:
the contact criterion includes an amplitude difference threshold and a phase difference threshold; and
reporting the candidate touch event is performed in accordance with a determination that: (i) a difference between the amplitude value and the baseline amplitude value satisfies the amplitude difference threshold, and (ii) a difference between the phase value and the baseline phase value satisfies the phase difference threshold.

15. The method of claim 1 , wherein:
the contact criterion includes a time threshold;
sensing the change in the baseline characteristics comprises sensing the change for a period of time; and
reporting the candidate touch event is performed in accordance with a determination that the period of time satisfies the time threshold.

16. The method of claim 1 , wherein the candidate touch event is selected from the group consisting of: a tap gesture, a press-and-hold gesture, a multi-tap gesture, a swipe gesture, a drag gesture, a pinch gesture, a pull gesture, a hover, and a twist gesture.

17. The method of claim 1 , wherein the separate electronic device is an artificial-reality system selected from the group consisting of: an augmented-reality system, a virtual-reality system, and a mixed-reality system.

18. A wearable device comprising:
one or more transducers;
one or more processors; and
memory storing one or more programs, which when executed by the one or more processors cause the wearable device to:
receive, by the one or more transducers, a set of signals that establish a signal pathway to the wearable device;
while receiving the set of signals:
determine baseline characteristics for the signal pathway; and
sense a change in the baseline characteristics caused by user interaction with an affordance of a projected user interface; and

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway satisfies a contact criterion, report a candidate touch event to a separate electronic device, wherein the separate electronic device creates the user interface or is in communication with another electronic device that creates the user interface.

19. A non-transitory computer-readable storage medium storing one or more programs configured for execution by one or more processors of a wearable device having one or more transducers, the one or more programs including instructions, which when executed by the one or more processors, cause the wearable device to:
receive, by the one or more transducers, a set of signals that establish a signal pathway to the wearable device;
while receiving the set of signals:
determine baseline characteristics for the signal pathway; and
sense a change in the baseline characteristics caused by user interaction with an affordance of a projected user interface; and

in accordance with a determination that the sensed change in the baseline characteristics for the signal pathway

Assignees

• Meta Platforms Tech Llc

Inventors

• Keller Sean Jason
• Trutna Tristan Thomas
• Benko Hrvoje

Classifications

• G06F3/016 Primary

  Input arrangements with force or tactile feedback as computer generated output to the user · CPC title

• G06F3/0426

  tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/012

  Head tracking input arrangements · CPC title

• G06F3/014

  Hand-worn input/output arrangements, e.g. data gloves · CPC title