Motion modeling in visual tracking

Patent metadata

Field	Value
Publication number	US-9767609-B2
Application number	US-201414179384-A
Country	US
Kind code	B2
Filing date	Feb 12, 2014
Priority date	Feb 12, 2014
Publication date	Sep 19, 2017
Grant date	Sep 19, 2017

Abstract

Embodiments are disclosed that relate to determining a pose of a device. One disclosed embodiment provides a method comprising receiving sensor information from one or more sensors of the device, and selecting a motion-family model from a plurality of different motion-family models based on the sensor information. The method further comprises providing the sensor information to the selected motion-family model and outputting an estimated pose of the device according to the selected motion-family model.

First claim

The invention claimed is:

1. A method of assessing a pose of a device, the method comprising:
receiving, from one or more sensors of the device, sensor information including a change in location of the device along three orthogonal axes and a change in orientation of the device along the three orthogonal axes;
using a pre-configured motion behavior classifier to determine a physical state of a user of the device based on the change in location of the device and the change in orientation of the device;
selecting a motion-family model from a plurality of different pre-configured motion-family models based on the physical state of the user;
providing the sensor information to the selected motion-family model;
receiving an estimated pose of the device from the selected motion-family model, wherein the selected motion-family model interprets the sensor information in accordance with a family of motion associated with the device for the selected motion-family model, the family of motion including an acceleration profile for the device, and the selected motion-family model matching one or more measurements of the device to a selected trajectory of the device; and
displaying content on the device based on the estimated pose of the device.

2. The method of claim 1 , wherein the device is a head-mounted display device configured to output the displayed content to a wearer of the device, the displayed content being mapped, based on the estimated pose, to changing locations of a real-world environment viewable through the head-mounted display device as the location and orientation of the head-mounted display device changes.

3. The method of claim 2 , further comprising adjusting one or more display properties for the displayed content based on the estimated pose of the device.

4. The method of claim 1 , wherein outputting the estimated pose of the device further comprises providing the sensor information and an output of the selected motion-family model to a pose filter and outputting the estimated pose of the device from the pose filter.

5. The method of claim 1 , wherein the sensor information indicates a physical state of a wearer of the device, the physical state indicating one or more of a position, orientation, and movement characteristic of the wearer.

6. The method of claim 1 , wherein selecting the motion-family model comprises classifying a family of motion exhibited by the device and comparing the classified family of motion to one or more families of motion associated with the plurality of different pre-configured motion-family models.

7. The method of claim 6 , wherein classifying the family of motion exhibited by the device comprises determining the family of motion based on a previously-output estimated pose of the device.

8. A head-mounted display device comprising:
a sensor;
a logic machine; and
a storage machine, the storage machine holding instructions executable by the logic machine to:
receive, from the sensor, sensor information including a change in location of the head-mounted display device along three orthogonal axes and a change in orientation of the head-mounted display device along the three orthogonal axes;
use a pre-configured motion behavior classifier to determine a physical state of a user of the head-mounted display device based on the change in location of the head-mounted display device and the change in orientation of the head-mounted display device;
select a motion-family model from a plurality of pre-configured motion-family models based on the physical state of a wearer of the head-mounted display device, each of the plurality of pre-configured motion-family models associated with a respective family of motion for the device, each respective family of motion including a different acceleration profile, and each pre-configured motion-family model matching one or more measurements of the sensor to a different trajectory of head movement indicating predefined expected movement of the head-mounted display device for the associated physical state of the user;
provide the sensor information and an output of the selected motion-family model to a pose filter;
output, from the pose filter, an estimated position and orientation of the head-mounted display device based on the sensor information and the selected motion-family model; and
display content on the head-mounted display device based on the estimated pose of the head-mounted display device.

9. The head-mounted display device of claim 8 , wherein the physical state of the wearer is determined based on the sensor information from the sensor.

10. The head-mounted display device of claim 8 , wherein the sensor information includes one or more measurements of a position and/or orientation of a head of the wearer.

11. The head-mounted display device of claim 10 , wherein the motion-family model matches the one or more measurements to a selected trajectory of head movement based on the physical state of the wearer.

12. The head-mounted display device of claim 8 , wherein the pose filter includes a Kalman filter.

13. The head-mounted display device of claim 8 , wherein selecting the motion-family model from the plurality of pre-configured motion-family models further comprises sending the sensor information and a previously-output estimated position and orientation to a pre-configured motion behavior classifier of the head-mounted display device.

14. The head-mounted display device of claim 13 , wherein the sensor information is provided to the pose filter and the pre-configured motion behavior classifier.

15. The head-mounted display device of claim 13 , wherein the pre-configured motion behavior classifier is used to analyze the sensor information and the previously-output estimated position and orientation to select the motion-family model.

16. The head-mounted display device of claim 15 , further comprising changing the motion-family model if the selected motion-family model is different than a current motion-family model.

17. A hardware memory device holding instructions executable by a logic machine to:
collect sensor information from one or more sensors of a head-mounted display;
send the sensor information to a pre-configured motion behavior classifier for determining a selected motion-family model from a plurality of different pre-configured motion-family models based on the sensor information;
in response to determining that the selected motion-family model is different than a previously-used motion-family model, replace the previously-used motion-family model with the selected motion-family model;
wherein each of the previously-used motion-family model and the selected motion-family model is used to adjust the sensor information to match one or more measurements indicating head movement to a selected trajectory of head movement for that motion-family model;
receive a time-varying pose estimate that is output based on an output of both the previously-used motion-family model and the selected motion-family model; and
display content on the head-mounted display based on the time-varying pose estimate of the head-mounted display.

18. The hardware memory device of claim 17 , wherein a first, earlier time-varying pose estimate is based on the previously-used motion-family model and a second, later time-varying pose estimate is based on the selected motion-family model.

19. The hardware memory device of claim 17 , wherein a first, earlier time-varying pose estimate is based on the previously-used motion-family model and a second, later time-varying p

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Eade Ethan
• Ebstyne Michael John
• Schaffalitzky Frederick
• Steedly Drew

Classifications

• G02B27/017

  Head mounted · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/012 Primary

  Head tracking input arrangements · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title