Apparatus, system, and method for wrist tracking and gesture detection via time of flight sensors

Patent metadata

Field	Value
Publication number	US-11150737-B2
Application number	US-202016820148-A
Country	US
Kind code	B2
Filing date	Mar 16, 2020
Priority date	Mar 16, 2020
Publication date	Oct 19, 2021
Grant date	Oct 19, 2021

Abstract

The disclosed wrist-tracking apparatus includes (1) a wristband dimensioned to be donned on a wrist of a user of an artificial reality system and (2) a set of Time of Flight (ToF) sensors coupled to the wristband, wherein each of the ToF sensors comprises (A) an emitter that emits a modulated pulse of energy, (B) a receiver that facilitates detecting a reflection of the modulated pulse of energy, and (C) a processing circuit communicatively coupled to the emitter and the receiver, wherein the processing circuit calculates a time of flight for the modulated pulse of energy based at least in part on the modulated pulse of energy and the reflection. Various other apparatuses, systems, and methods are also disclosed.

First claim

What is claimed is:

1. A wrist-tracking apparatus comprising:
a wristband dimensioned to be donned on a wrist of a user of an artificial reality system;
a set of Time of Flight (ToF) sensors coupled to the wristband, wherein each of the ToF sensors comprises:
an emitter that emits a modulated pulse of energy;
a receiver that facilitates detecting a reflection of the modulated pulse of energy; and
a processing circuit communicatively coupled to the emitter and the receiver, wherein the processing circuit calculates a time of flight for the modulated pulse of energy by measuring a time differential between the emission of the modulated pulse of energy and the detection of the reflection; and

at least one physical processor communicatively coupled to the ToF sensors, wherein the physical processor:
implements a heuristic model to generate a 2 Degree of Freedom (2-DoF) representation of a current angle of the wrist of the user based at least in part on times of flight calculated by the ToF sensors and distances between the ToF sensors;
detects a gesture made by the user based at least in part on the 2-DoF representation of the current angle of the wrist of the user;
generates, in response to detecting the gesture made by the user, an input command for the artificial reality system that accounts for the gesture; and
facilitates modifying at least one virtual component of the artificial reality system to account for the gesture based at least in part on the input command.

2. The wrist-tracking apparatus of claim 1 , wherein:
the set of ToF sensors comprises:
a first ToF sensor coupled to one side of the wristband; and
a second ToF sensor coupled to an opposing side of the wristband;

the physical processor determines the current angle of the wrist of the user based at least in part on:
a first time of flight of a first modulated pulse of energy emitted by the first ToF sensor; and
a second time of flight of a second modulated pulse of energy emitted by the second ToF sensor.

3. The wrist-tracking apparatus of claim 2 , wherein the first time of flight of the first modulated pulse of energy is at least one of:
indefinite due at least in part to a failure to detect a reflection of the first modulated pulse of energy; or
definite due at least in part to a successful detection of a reflection of the first modulated pulse of energy.

4. The wrist-tracking apparatus of claim 2 , wherein the physical processor detects the gesture made by the user based at least in part on the first time of flight and the second time of flight.

5. The wrist-tracking apparatus of claim 4 , wherein the physical processor implements the heuristic model to generate a 2 Degree of Freedom (DoF) representation of the current angle of the user's wrist based at least in part on the first time of flight and the second time of flight.

6. The wrist-tracking apparatus of claim 4 , further comprising at least one Inertial Measurement Unit (IMU) communicatively coupled to the physical processor, wherein the physical processor:
receives orientation data from the IMU; and
detects the gesture made by the user based at least in part on the first time of flight, the second time of flight, and the orientation data from the IMU.

7. The wrist-tracking apparatus of claim 6 , wherein the gesture comprises at least one of:
a pinching motion made by fingers of the user;
a hand motion made by a hand of the user; or
a wrist motion made by the wrist of the user.

8. The wrist-tracking apparatus of claim 4 , further comprising a set of modulated proximity sensors communicatively coupled to the physical processor, wherein the physical processor:
receives light-intensity data from the set of modulated proximity sensors; and
detects the gesture made by the user based at least in part on the first time of flight, the second time of flight, and the light-intensity data from the set of modulated proximity sensors.

9. The wrist-tracking apparatus of claim 1 , wherein:
the gesture comprises at least one of:
an upward motion made by the wrist of the user;
a downward motion made by the wrist of the user;
a leftward motion made by the wrist of the user; or
a rightward motion made the wrist of the user; and

the physical processor performs, based at least in part on the gesture, a calibration process that accounts for at least one drift error to ensure that readings of the current angle of the user's wrist remain accurate throughout an operating session.

10. The wrist-tracking apparatus of claim 1 , wherein the ToF sensors are coupled to the wristband such that the emitter and the receiver are oriented toward a wrist of the user.

11. The wrist-tracking apparatus of claim 1 , wherein the modulated pulse of energy comprises at least one of:
a modulated pulse of electromagnetic radiation; or
a modulated pulse of sound energy.

12. An artificial reality system comprising:
a head-mounted display; and
a wearable device communicatively coupled to the head-mounted display, the wearable device comprising:
a set of Time of Flight (ToF) sensors coupled to the wearable device, wherein each of the ToF sensors comprises:
an emitter that emits a modulated pulse of energy;
a receiver that facilitates detecting a reflection of the modulated pulse of energy; and
a processing circuit communicatively coupled to the emitter and the receiver, wherein the processing circuit calculates a time of flight for the modulated pulse of energy by measuring a time differential between the emission of the modulated pulse of energy and the detection of the reflection; and

at least one physical processor communicatively coupled to the ToF sensors, wherein the physical processor:
implements a heuristic model to generate a 2 Degree of Freedom (2-DoF) representation of a current angle of a wrist of a user based at least in part on times of flight calculated by the ToF sensors and distances between the ToF sensors;
detects a gesture made by the user based at least in part on the 2-DoF representation of the current angle of the wrist of the user;
generates, in response to detecting the gesture made by the user, an input command for the artificial reality system that accounts for the gesture; and
facilitates modifying at least one virtual component of the artificial reality system to account for the gesture based at least in part on the input command.

13. The artificial reality system of claim 12 , wherein:
the set of ToF sensors comprises:
a first ToF sensor coupled to one side of the wearable device wristband; and
a second ToF sensor coupled to an opposing side of the wearable device; and

the physical processor determines the current angle of the wrist of the user based at least in part on:
a first time of flight of a first modulated pulse of energy emitted by the first ToF sensor; and
a second time of flight of a second modulated pulse of energy emitted by the second ToF sensor.

14. The artificial reality system of claim 13 , wherein the first time of flight of the first modulated pulse of energy is at least one of:
indefinite due at least in part to a failure to detect a reflection of the first modulated pulse of energy; or
definite due at least in part to a successful detection of a reflection of the first modulated pulse of energy.

15. The artificial reality system of claim 13 , wherein the physical processor detects a gesture made by the user based at least in part on the first time of flight and the second time of flight.

16. The artificial reality system of claim 15 , wherein the physical processor generates, in r

Assignees

• Facebook Tech Llc

Inventors

• Parizi Farshid Salemi
• Kienzle Wolf
• Whitmire Eric Michael

Classifications

• G06V40/28

  Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113) · CPC title

• G01S15/872

  Combination of several systems for attitude determination (using inertial means G01C9/00, control of attitude G05D1/49) · CPC title

• G06F3/014

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G01S17/58

  Velocity or trajectory determination systems; Sense-of-movement determination systems · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title