Tracking sensor integration system and method for recursive estimation of pose of user's body part

Patent metadata

Field	Value
Publication number	US-10705606-B1
Application number	US-201816001849-A
Country	US
Kind code	B1
Filing date	Jun 6, 2018
Priority date	Oct 23, 2017
Publication date	Jul 7, 2020
Grant date	Jul 7, 2020

Abstract

A tracking sensor integration system presented herein collects sensor data obtained for each time frame by a plurality of sensors attached to a wearable garment placed on a user's hand. A controller coupled to the tracking sensor integration system calculates a measurement gain based at least in part on collected sensor data, and determines prediction for a pose of the user's hand for the current time frame using the collected sensor data and a plurality of estimation parameters for the current time frame. The controller then updates the estimation parameters for the current time frame, based in part on the measurement gain and the prediction for the pose of the user's hand. The controller determines an estimated pose for the user's hand, based in part on the updated estimation parameters and the collected sensor data.

First claim

What is claimed is:

1. A tracking sensor integration system comprising:
a plurality of sensors attached to a wearable garment placed on a user's hand, the plurality of sensors configured to obtain sensor data during a plurality of time frames; and
a controller interfaced with the plurality of sensors, the controller configured, for each time frame of the plurality of time frames, to:
update a measurement gain for a current time frame of the plurality of time frames based on a portion of the sensor data collected for the current time frame and a time frame preceding the current time frame and a value of the measurement gain for the time frame preceding the current time frame,
determine prediction for a pose of the user's hand for the current time frame using sensor data collected for the current time frame and a plurality of estimation parameters for the current time frame,

update the estimation parameters for the current time frame, based in part on the updated measurement gain and the prediction for the pose of the user's hand, the estimation parameters representing weighted regressors describing a direct relation between the sensor data and the pose of the user's hand, and
determine an estimated pose for the user's hand for the current time frame, based in part on the updated estimation parameters and the sensor data collected for the current time frame.

2. The system of claim 1 , wherein the controller is further configured to:
update a measurement covariance based in part on the measurement gain and the sensor data collected for the current time frame; and
update the measurement gain for a time frame following the current time frame, based on the updated measurement covariance and sensor data collected by the plurality of sensors during the time frame following the current time frame.

3. The system of claim 1 , wherein the controller is further configured to:
determine a plurality of weights based on a defined accuracy of estimating the pose of the user's hand;
apply the weights to the sensor data collected for the current time frame to obtain weighed sensor data; and
calculate the measurement gain based in part on the weighted sensor data.

4. The system of claim 1 , wherein the controller is further configured to:
determine measurement data related to the pose of the user's hand using the sensor data collected for the current time frame;
calculate a difference between the measurement data and the prediction for the pose of the user's hand; and
update the estimation parameters for the current time frame based on the measurement gain and the difference.

5. The system of claim 1 , wherein the controller is further configured to:
provide information about the estimated pose for the user's hand to a console for updating a visual presentation of the user's hand on an electronic display of a head-mounted display.

6. The system of claim 1 , wherein the controller is further configured to:
replace, in a memory coupled to the controller, information about the estimation parameters for the time frame preceding the current time frame with information about the updated estimation parameters for the current time frame; and
replace, in the memory, information about sensor data collected by the plurality of sensors elements during the time frame preceding the current time frame with information about the sensor data collected for the current time frame.

7. The system of claim 1 , wherein the controller is further configured to:
determine a vector of regressors for the current time frame based on the sensor data collected for the current time frame; and
determine the estimated pose for the user's hand, based on the updated estimation parameters and the vector of regressors.

8. The system of claim 7 , wherein the controller is further configured to:
determine the prediction for the pose of the user's hand for the current time frame, based on the vector of regressors for the current time frame and the estimation parameters for the current time frame.

9. The system of claim 1 , wherein the plurality of sensors are selected from a group consisting of optical sensors, magnetic sensing sensors, radio frequency (RF) based sensors, a position sensor, and an inertial measurement unit (IMU).

10. A tracking sensor integration system, the system configured to:
obtain sensor data during a plurality of time frames by a plurality of sensors attached to a wearable garment placed on a portion of a user's body;
update a measurement gain for a current time frame of the plurality of time frames based on a portion of the sensor data collected for the current time frame and a time frame preceding the current time frame and a value of the measurement gain for the time frame preceding the current time frame;
determine prediction for a pose of the portion of the user's body for the current time frame using sensor data collected for the current time frame and a plurality of estimation parameters for the current time frame;
update the estimation parameters for the current time frame, based in part on the updated measurement gain and the prediction for the pose of the portion of the user's body, the estimation parameters representing weighted regressors describing a direct relation between the sensor data and the pose of the user's hand; and
determine an estimated pose for the portion of the user's body for the current time frame, based in part on the updated estimation parameters and the sensor data collected for the current time frame.

11. The system of claim 10 , wherein the system is further configured to:
update a measurement covariance based in part on the measurement gain and the sensor data collected for the current time frame; and
update the measurement gain for a time frame following the current time frame, based on the updated measurement covariance and sensor data collected by the plurality of sensors during the time frame following the current time frame.

12. The system of claim 10 , wherein the system is further configured to:
determine measurement data related to the pose of the portion of the user's body using the sensor data collected for the current time frame;
calculate a difference between the measurement data and the prediction for the pose of the portion of the user's body; and
update the estimation parameters for the current time frame based on the measurement gain and the difference.

13. The system of claim 10 , wherein the system is further configured to:
replace, in a memory of the system, information about the estimation parameters for the time frame preceding the current time frame with information about the updated estimation parameters for the current time frame; and
replace, in the memory, information about sensor data collected by the plurality of sensors elements during the time frame preceding the current time frame with information about the sensor data collected for the current time frame.

14. The system of claim 10 , wherein the system is further configured to:
determine a vector of regressors for the current time frame based on the sensor data collected for the current time frame; and
determine the estimated pose for the portion of the user's body, based on the updated estimation parameters and the vector of regressors.

15. The system of claim 14 , wherein the system is further configured to:
determine the prediction for the pose of the portion of the user's body for the current time frame, based on the vector of regressors for the current time frame and the estimation parameters for the current time frame.

16. A method comprising:
obtaining sen

Assignees

• Facebook Tech Llc

Inventors

• Colonnese Nicholas
• Clark Justin
• Keller Sean Jason
• Trutna Tristan Thomas

Classifications

• G06T2207/30204

  Marker · CPC title

• G06T7/248

  involving reference images or patches · CPC title

• G06F3/014 Primary

  Hand-worn input/output arrangements, e.g. data gloves · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title