Use of comparative sensor data to determine orientation of head relative to body

Patent metadata

Field	Value
Publication number	US-9268136-B1
Application number	US-201213631454-A
Country	US
Kind code	B1
Filing date	Sep 28, 2012
Priority date	Sep 28, 2012
Publication date	Feb 23, 2016
Grant date	Feb 23, 2016

Abstract

Methods and systems are described that involve a head-mountable display (HMD) or an associated device determining the orientation of a person's head relative to their body. To do so, example methods and systems may compare sensor data from the HMD to corresponding sensor data from a tracking device that is expected to move in a manner that follows the wearer's body, such a mobile phone that is located in the HMD wearer's pocket.

First claim

We claim:

1. A computer-implemented method comprising:
detecting, by a computing device, sensor data that is indicative of an association between movement of a tracking device and body movement;
in response to detecting the sensor data that is indicative of the association between movement of the tracking device and body movement:
determining a direction of the body movement that is associated with the movement of the tracking device;
determining, based on the direction of the body movement, a forward-backward body axis of a body; and
determining a base orientation of a tracking device relative to the forward-backward body axis;

determining a first orientation of a head-mountable device (HMD) relative to the tracking device, wherein the HMD and the tracking device are separate devices; and
determining a first head orientation relative to the body based on both: (a) the first orientation of the HMD relative to the tracking device and (b) the base orientation of the tracking device relative to the forward-backward body axis, wherein determining the first head orientation relative to the body comprises offsetting the first orientation of the HMD relative to the tracking device by the base orientation of the tracking device relative to the forward-backward body axis.

2. The method of claim 1 , wherein the computing device is the HMD.

3. The method of claim 1 , wherein the computing device is the tracking device.

4. The method of claim 1 , wherein the tracking device is a mobile phone.

5. The method of claim 1 , wherein detecting the sensor data that is indicative of the association between movement of the tracking device and body movement comprises:
receiving sensor data from at least one of the HMD and the tracking device, wherein the sensor data is indicative of movement; and
determining that the sensor data is characteristic of movement along the forward-backward body axis.

6. The method of claim 5 , wherein determining that the sensor data is characteristic of movement along the forward-backward body axis comprises determining that the sensor data is characteristic of walking.

7. The method of claim 5 , wherein the sensor data comprises at least one of: (a) gyroscope data associated with the HMD, (b) accelerometer data associated with the HMD, (c) gyroscope data associated with the tracking device, and (d) accelerometer data associated with the tracking device.

8. The method of claim 1 , wherein determining a forward-backward body axis of a body comprises determining a direction of forward body movement.

9. The method of claim 1 , wherein determining a base orientation of the tracking device relative to the forward-backward body axis comprises determining an angle between a forward-facing direction of the tracking device and the directional component along the forward-backward body axis.

10. The method of claim 1 , wherein the first orientation of the HMD relative to the tracking device is determined based on both: (a) magnetometer data associated with the HMD and (b) magnetometer data associated with the tracking device.

11. The method of claim 10 , wherein determining the first orientation of the HMD relative to the tracking device comprises:
determining a first orientation of the HMD relative to magnetic north based on the magnetometer data associated with the HMD;
determining a first orientation of the tracking device relative to magnetic north based on the magnetometer data associated with the tracking device; and
determining the first orientation of the HMD relative to the tracking device based on a difference between (a) the first orientation of the HMD relative to magnetic north and (b) the first orientation of the tracking device relative to magnetic north.

12. The method of claim 1 , further comprising:
subsequently detecting a calibration event, wherein detecting the calibration event comprises receiving an indication that the tracking device has moved in relation to the body; and
in response to detecting the calibration event:
analyzing the sensor data to detect subsequent sensor data that is indicative of an association between second movement of a tracking device and second body movement; and
in response to detecting the subsequent sensor data that is indicative of the association, re-determining, based on a direction of the second body movement, the base orientation of the tracking device relative to the forward-backward body axis.

13. The method of claim 1 , further comprising causing a display of the HMD to display graphics based on the first head orientation relative to the body.

14. The method of claim 1 , further comprising initiating a computing action based on the first head orientation relative to the body.

15. The method of claim 1 , wherein determining a first head orientation relative to the body comprises determining a rotation of the head relative to the forward-backward body axis.

16. The method of claim 1 , wherein determining a first head orientation relative to the body comprises determining two or more of: (a) a rotation of the head relative to the forward-backward body axis, (b) a pitch of the head relative to an upward-downward body axis and (c) a yaw of the head relative to the forward-backward body axis and the upward-downward body axis.

17. A non-transitory computer readable medium having stored therein instructions that are executable to cause a computing device to perform functions comprising:
detecting sensor data that is indicative of an association between movement of a tracking device and body movement;
in response to detecting the sensor data that is indicative of the positional association between movement of the tracking device and body movement:
determining a direction of the body movement that is associated with the movement of the tracking device;
determining a forward-backward body axis of a body based on the direction of the body movement; and
determining a base orientation of a tracking device relative to the forward-backward body axis;

determining a first orientation of a head-mountable device (HMD) relative to the tracking device, wherein the HMD and the tracking device are separate devices; and
determining a first head orientation relative to the body based on both: (a) the first orientation of the HMD relative to the tracking device and (b) the base orientation of the tracking device relative to the forward-backward body axis, wherein determining the first head orientation relative to the body comprises offsetting the first orientation of the HMD relative to the tracking device by the base orientation of the tracking device relative to the forward-backward body axis.

18. The non-transitory computer readable medium of claim 17 , wherein the first orientation of the HMD relative to the tracking device is determined based on both: (a) magnetometer data associated with the HMD and (b) magnetometer data associated with the tracking device.

19. The non-transitory computer readable medium of claim 18 , wherein determining the first orientation of the HMD relative to the tracking device comprises:
determining a first orientation of the HMD relative to magnetic north based on the magnetometer data associated with the HMD;
determining a first orientation of the tracking device relative to magnetic north based on the magnetometer data associated with the tracking device; and
determining the first orientation of the HMD relative to the tracking device based on a difference between (a) the first orientation of the HMD relative to magnetic north and (b) the first o

Assignees

• Google Inc

Inventors

• Starner Thad Eugene
• Johnson Michael Patrick

Classifications

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G01P15/00

  Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B27/0179

  Display position adjusting means not related to the information to be displayed · CPC title

• G01C19/00

  Gyroscopes; Turn-sensitive devices using vibrating masses; Turn-sensitive devices without moving masses; Measuring angular rate using gyroscopic effects · CPC title