Armband for tracking hand motion using electrical impedance measurement

Patent metadata

Field	Value
Publication number	US-10481699-B2
Application number	US-201715661317-A
Country	US
Kind code	B2
Filing date	Jul 27, 2017
Priority date	Jul 27, 2017
Publication date	Nov 19, 2019
Grant date	Nov 19, 2019

Abstract

A system includes a wearable device including sensors arranged at different locations on the wearable device. Each sensor measures electrical signals transmitted from a wrist or arm of a user. A position computation circuit is coupled to the sensors. The position computation circuit computes, using information derived from the electrical signals with a machine learning model, an output that describes a hand position of a hand of the wrist or arm of the user.

First claim

The invention claimed is:

1. A system, comprising:
a wearable device, comprising sensors arranged in a two-dimensional pattern on the wearable device, each sensor configured to measure electrical signals transmitted through a wrist or arm of a user, the electrical signals comprising information of values of an electrical impedance; and
a position computation circuit coupled to the sensors, the position computation circuit configured to use a machine learning model to receive the values of the electrical impedance derived from the electrical signals measured by the sensors arranged in the two-dimensional pattern and transmitted from the wrist or arm of the user to compute an output that describes a hand position of a hand of the wrist or arm of the user.

2. The system of claim 1 , wherein the position computation circuit is located on or within the wearable device.

3. The system of claim 1 , wherein the position computation circuit is located on or within a host system external to the wearable device.

4. The system of claim 1 , wherein each sensor comprises an electrode and a conductive agent located between the electrode and the wrist or arm of the user, each sensor further configured to transmit an alternating current (AC) signal, a direct current (DC) signal, or a wide-bandwidth AC signal including a plurality of frequencies into the wrist or arm of the user.

5. The system of claim 1 , wherein each electrical signal comprises one of:
a voltage and a phase of the voltage;
an electric current and a phase of the electric current; or
a magnitude of a DC signal.

6. The system of claim 1 , wherein the two-dimensional pattern is a grid array or a checkerboard pattern.

7. The system of claim 1 , wherein the machine learning model is configured to use information derived from the electrical signals to compute the output, the information comprises one or more of:
aggregate values based on the electrical impedance, the electrical impedance measured between each pair of the sensors, the electrical impedance determined based on probe signals transmitted into the wrist or arm of the user by the pair of sensors and the electrical signals;
a shape of a wave of the electrical signals;
a frequency-domain representation of the electrical signals; and
a time-domain sample of the electrical signals.

8. The system of claim 1 , wherein the position computation circuit is further configured to extract features from information derived from the electrical signals, the features comprising one or more of:
angles between joints defining the hand position of a hand of the wrist or arm of the user; and
a reduced representation of a change in the hand position of a hand of the wrist or arm of the user, the reduced representation defining a difference between a present hand position of the hand of the wrist or arm of the user and a previous hand position of the hand of the wrist or arm of the user.

9. The system of claim 1 , wherein each sensor is further configured to transmit a probe signal into the wrist or arm of the user by varying one or more of:
a time period of the probe signal;
an amplitude of the probe signal; and
a phase of the probe signal.

10. The system of claim 1 , wherein a first sensor is configured to transmit a probe signal into the wrist or arm of the user by staggering transmission of the probe signal with respect to transmission of other probe signals by other sensors.

11. The system of claim 1 , further comprising:
an inertial measurement unit configured to generate inertial signals corresponding to movement of the wearable device and the user's arm, wherein the position computation circuit is further configured to compute, using information derived from the inertial signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

12. The system of claim 11 , wherein the inertial measurement unit comprises one or more of:
a gyroscope;
an accelerometer; and
a magnetometer.

13. The system of claim 1 , further comprising:
one or more cameras configured to generate image signals of the hand of the wrist or arm of the user from one or more angles, wherein the position computation circuit is further configured to compute, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

14. The system of claim 13 , wherein one of the cameras is a depth camera, a Red-Green-Blue (RGB) camera, an Infrared camera, or a camera mounted on a head-mounted display (HMD).

15. The system of claim 1 , further comprising:
an inertial measurement unit configured to generate inertial signals corresponding to movement of the wearable device, wherein the position computation circuit is further configured to train the machine learning model to generate, using information derived from the inertial signals, the output that describes the hand position of the hand of the wrist or arm of the user.

16. The system of claim 1 , further comprising:
one or more cameras configured to generate image signals of the hand of the wrist or arm of the user, wherein the position computation circuit is further configured to train the machine learning model to generate, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand of the wrist or arm of the user.

17. The system of claim 1 , wherein the computed output further describes forces exerted by the hand of the wrist or arm of the user on objects touching the hand of the wrist or arm of the user.

18. The system of claim 1 , wherein the position computation circuit is further configured to:
receive image signals from a camera mounted on an HMD;
determine comparison signals by comparing the image signals with the computed output; and
transmit the comparison signals to a host system.

19. The system of claim 1 , wherein the computed output comprises parameters of a hand shape model, the parameters corresponding to:
a plurality of joints of the hand of the wrist or arm of the user;
a plurality of edges between pairs of the joints;
a plurality of angles between pairs of the edges;
a mesh comprising a plurality of vertices and for each vertex, a distance between the vertex and one or more joints.

20. An HMD, comprising:
a position computation circuit coupled to sensors arranged in a two-dimensional pattern on a wearable device, each sensor configured to measure electrical signals transmitted from a wrist or arm of a user, the electrical signals comprising information of values of an electrical impedance, the position computation circuit configured to use a machine learning model to receive the values of the electrical impedance derived from the electrical signals measured by the sensors arranged in the two-dimensional pattern and transmitted from the wrist or arm of the user to compute an output that describes a hand position of a hand of the wrist or arm of the user; and
a display panel configured to receive the computed output from the position computation circuit.

21. The HMD of claim 20 , further comprising:
a camera configured to generate image signals of the hand of the wrist or arm of the user, wherein the position computation circuit is further configured to compute, using information derived from the image signals with the machine learning model, the output that describes the hand position of the hand

Assignees

• Facebook Tech Llc

Inventors

• Mu Beipeng
• De Nardi Renzo
• Newcombe Richard Andrew
• King Raymond
• Gander Evan Paul
• Wang Robert Y

Classifications

• G06F3/0308

  comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen · CPC title

• G06F3/015

  Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06T7/74

  involving reference images or patches · CPC title