Identification of Facial Expression of Head-Mountable Display Wearer

Patent metadata

Field	Value
Publication number	US-2025022312-A1
Application number	US-202118577079-A
Country	US
Kind code	A1
Filing date	Jul 16, 2021
Priority date	Jul 16, 2021
Publication date	Jan 16, 2025
Grant date	—

Abstract

Facial movement of a wearer of a head-mountable display (HMD) is detected by detecting movement of a gasket of the HMD positioned against a face of the wearer using resistive or magnetic sensors within the gasket. A facial expression of the wearer is identified based on the detected facial movement.

First claim

We claim:

1 . A non-transitory computer-readable data storage medium storing program code executable by a processor to perform processing comprising:
detecting facial movement of a wearer of a head-mountable display (HMD) by detecting movement of a gasket of the HMD positioned against a face of the wearer using a plurality of resistive or magnetic sensors within the gasket; and identifying a facial expression of the wearer based on the detected facial movement.

2 . The non-transitory computer-readable data storage medium of claim 1 , wherein the processing further comprises:
performing an action related to the wearer of the HMD based on the identified facial expression of the wearer.

3 . The non-transitory computer-readable data storage medium of claim 1 , wherein detecting the movement of the gasket of the HMD using the plurality of resistive or magnetic sensors comprises:
detecting planar movement of the gasket along an external surface of the gasket in contact with the face of the wearer.

4 . The non-transitory computer-readable data storage medium of claim 3 , wherein detecting the movement of the gasket of the HMD using the plurality of resistive or magnetic sensors further comprises:
detecting axial movement of the gasket perpendicular to the external surface of the gasket in contact with the face of the wearer.

5 . The non-transitory computer-readable data storage medium of claim 1 , wherein the resistive or magnetic sensors are resistive strain gauge sensors, and wherein detecting the movement of the gasket of the HMD comprises:
receiving sensor values from the resistive strain gauge sensors corresponding to physical strain on the gasket as the wearer is currently exhibiting the facial expression as compared to baseline sensor values corresponding to the physical strain on the gasket when the wearer is exhibiting a relaxed neutral facial expression, wherein the facial expression of the wearer is determined based on the received sensor values.

6 . The non-transitory computer-readable data storage medium of claim 1 , wherein the resistive or magnetic sensors are magnetic Hall effect sensors, and wherein detecting the movement of the gasket of the HMD comprises:
receiving sensor values from the magnetic Hall effect sensors corresponding to physical displacement of the gasket as the wearer is currently exhibiting the facial expression as compared to baseline sensor values corresponding to the physical displacement of the gasket when the wearer is exhibiting a relaxed neutral facial expression, wherein the facial expression of the wearer is determined based on the received sensor values.

7 . The non-transitory computer-readable data storage medium of claim 1 , wherein the processing further comprises:
receiving sensor values from the resistive or magnetic sensors when the wearer is exhibiting a relaxed neutral facial expression; and determining whether the HMD is properly fitted to the wearer within a threshold by comparing the sensor values to baseline sensor values before the wearer put on the HMD.

8 . The non-transitory computer-readable data storage medium of claim 1 , wherein the processing further comprises:
receiving sensor values from the resistive or magnetic sensors before the wearer put on the HMD; and determining whether the gasket of the HMD has become worn by more than a threshold by comparing the sensor values to baseline sensor values prior to first use of the HMD.

9 . A method comprising:
receiving, by a processor, sensor values from a plurality of resistive or magnetic sensors within a gasket of a head-mountable display (HMD) positioned against a face of a wearer of the HMD, as the wearer is exhibiting a facial expression; identifying, by the processor, the facial expression of the wearer based on the received sensor values; and performing, by the processor, an action related to the wearer based on the identified facial expression of the wearer.

10 . The method of claim 9 , wherein the resistive or magnetic sensors are resistive strain gauge sensors, and the sensor values are indicative of physical strain on the gasket resulting from movement of the gasket as the wearer is exhibiting the facial expression.

11 . The method of claim 9 , wherein the resistive or magnetic sensors comprise magnetic Hall effect sensors, and the sensor values are indicative of physical displacement of the gasket resulting from movement of the gasket as the wearer is exhibiting the facial expression.

12 . A head-mountable display (HMD) comprising:
a gasket positionable against a face of a wearer; and a plurality of resistive or magnetic sensors within the gasket to detect movement of the gasket responsive to facial movement of the wearer while the wearer is exhibiting a facial expression.

13 . The HMD of claim 12 , further comprising:
circuitry to identify the facial expression of the wearer based on the detected movement of the gasket.

14 . The HMD of claim 12 , wherein the resistive or magnetic sensors comprise resistive strain gauge sensors that measure physical strain on the gasket resulting from the movement of the gasket responsive to the facial movement of the wearer.

15 . The HMD of claim 12 , wherein the resistive or magnetic sensors comprise magnetic Hall effect sensors that measure physical displacement of the gasket resulting from the movement of the gasket responsive to the facial movement of the wearer.

Assignees

• Hewlett Packard Development Co

Inventors

• Mazurkiewicz Paul Howard
• Nouri Joseph
• Lessman Mark

Classifications

• G02B27/017

  Head mounted · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G01R33/07

  Hall effect devices · CPC title

• G01B7/16

  for measuring the deformation in a solid, e.g. by resistance strain gauge · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title