Mitigation of head-mounted-display impact via biometric sensors and language processing

Patent metadata

Field	Value
Publication number	US-10568573-B2
Application number	US-201715452354-A
Country	US
Kind code	B2
Filing date	Mar 7, 2017
Priority date	Mar 7, 2017
Publication date	Feb 25, 2020
Grant date	Feb 25, 2020

Abstract

Users of head-mounted displays (HMDs) often experience nausea and other symptoms often referred to a “virtual reality sickness.” Various health and biometric sensors are employed to gather baseline sensor measurements. The HMD or a device associated with it can then generate a range or threshold such that any future sensor measurements that fall outside of the range or cross the threshold suggest a health concern for the user of the HMD. The HMD can also be coupled to a microphone, and natural language processing (NLP) is employed to detect negative words or noises made by the user that could also suggest a health concern for the user. After detecting the health concern, the HMD can warn its user, can shut off certain functions, can communicate with an external device, or can trigger an alarm.

First claim

What is claimed is:

1. A method for illness mitigation during gameplay involving use of a head-mounted display (HMD), the method comprising:
receiving one or more baseline sensor measurement values from a health sensor under conditions designated as normal, the baseline sensor measurement values measuring a health characteristic of a user of the head-mounted display (HMD) during gameplay of a plurality of different types of games;
generating a health threshold value set based on the baseline sensor measurement values for the different types of games, wherein each different type of game is associated with a different health threshold value in the health threshold value set;
receiving game data indicating that the user is playing a game at an identified time period;
receiving an active sensor measurement value from the health sensor during gameplay of the game at the identified time period, wherein the active sensor measurement value measures the health characteristic of the user of the head-mounted display (HMD) following generation of the health threshold value;
identifying a type of the game being played at the identified time period indicated by the game data;
identifying which of the different health threshold values in the health threshold value set is associated with the identified type of the game;
identifying that the active sensor measurement value has reached or crossed the identified health threshold value associated with the identified type of the game; and
displaying a visual indicator on a display portion of the head-mounted display (HMD) in response to the identification that the active sensor measurement value has reached or crossed the identified health threshold value associated with the identified type of the game.

2. The method of claim 1 , further comprising activating an audio output device associated with the head-mounted display (HMD) to play an audible warning in response to the identification that the active sensor measurement value has reached or crossed the identified health threshold value.

3. The method of claim 1 , further comprising turning off one or more functions of the head-mounted display (HMD) in response to the identification that the active sensor measurement value has reached or crossed the identified health threshold value.

4. The method of claim 1 , further comprising sending a transmission to an external device in response to the identification of the health concern, wherein the transmission includes at least one of a text-based notification, an audio-based notification, or a visual notification.

5. The method of claim 1 , wherein the health sensor is a thermometer, and wherein the active sensor measurement value is a body temperature measurement value of the user of the head-mounted display (HMD).

6. The method of claim 1 , wherein the health sensor is at least one of a moisture sensor or a galvanic skin response sensor, and wherein the active sensor measurement value is a skin sweat measurement value of the user of the head-mounted display (HMD).

7. The method of claim 1 , wherein the health sensor is a camera, and wherein the active sensor measurement value concerns at least one eye of the user of the head-mounted display (HMD), and wherein the active sensor measurement value is at least one of a pupil dilation measurement value, an eye movement measurement value, or an eye moisture measurement.

8. The method of claim 1 , wherein the health sensor is at least one of a accelerometer or a gyroscope, and wherein the active sensor measurement value concerns at least one of an orientation or a position of the user of the head-mounted display (HMD).

9. The method of claim 1 , wherein the health sensor is a microphone, and wherein the active sensor measurement value corresponds to an attribute of a noise made by the user of the head-mounted display (HMD).

10. The method of claim 1 , wherein identifying that the active sensor measurement value has reached or crossed the health threshold value includes identifying that the active sensor measurement value has reached or exceeded the health threshold value.

11. The method of claim 1 , wherein identifying that the active sensor measurement value has reached or crossed the health threshold value includes identifying that the active sensor measurement value has reached or fallen below the health threshold value.

12. The method of claim 1 further comprising:
receiving new game data that includes a new active sensor measurement value during gameplay of a different game;
identifying that the new game data was obtained from a type of the different game, the identified type of the different game being different from the identified type of the game;
identifying which of the different health threshold values in the health threshold value set is associated with the different type of the different game;
identifying that the new active sensor measurement value has reached or crossed the health threshold value identified as associated with the different type of the different game; and
generating a notification on a display device associated with the HMD.

13. A head-mounted display (HMD) system for illness mitigation, the head-mounted display (HMD) system comprising:
a body to be worn on a head of a user;
a display integrated with the body;
a memory to store instructions;
a health sensor that provides one or more baseline sensor measurement values under conditions designated as normal, the baseline sensor measurement values measuring a health characteristic of the user during gameplay of a plurality of different types of games,
a processor coupled to the memory, wherein execution of the instructions by the processor causes the head-mounted display (HMD) system to:
generate a health threshold value set based on the baseline sensor measurement values for the different types of games, wherein each different type of game is associated with a different health threshold value in the health threshold value set,
receives game data indicating that the user is playing a game at an identified time period,
receive an active sensor measurement value from the health sensor during gameplay of the game at the identified time period, wherein the active sensor measurement value measures the health characteristic of the user of the head-mounted display (HMD) following generation of the health threshold value,
identify a type of the game being played at the identified time period indicated by the game data,
identify which of the different health threshold values in the health threshold value set is associated with the identified type of the game;
identify that the active sensor measurement value has reached or crossed the identified health threshold value associated with the identified type of the game, and
display a visual indicator on the display integrated with the body in response to the identification that the active sensor measurement value has reached or crossed the identified health threshold value associated with the identified type of the game.

14. The head-mounted display (HMD) system of claim 13 , further comprising a wireless communication interface, wherein the head-mounted display (HMD) system receives the baseline sensor measurement value and the active sensor measurement value from the health sensor wirelessly via the wireless communication interface.

15. The head-mounted display (HMD) system of claim 13 , further comprising an alarm system that includes at least one of an audio output device or a visual output device, wherein execution of the instructions by the processor causes the head-mounted display (HMD) system to activa

Assignees

• Sony Interactive Entertainment LLC

Inventors

• Campbell Kevin
• Cvetkovic Milan
• Garza Patricio
• Johnson Abinadi
• Corenbaum John
• Lau Quan

Classifications

• A61B5/6803 Primary

  Head-worn items, e.g. helmets, masks, headphones or goggles · CPC title

• A61B5/7405

  using sound · CPC title

• G09G5/003

  Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto (specific for a CRT G09G1/165; for a flat panel G09G3/2092) · CPC title

• G16H20/70

  relating to mental therapies, e.g. psychological therapy or autogenous training · CPC title

• A61B3/14

  Arrangements specially adapted for eye photography · CPC title