User experience localizing binaural sound during a telephone call

Patent metadata

Field	Value
Publication number	US-12593190-B2
Application number	US-202418424655-A
Country	US
Kind code	B2
Filing date	Jan 26, 2024
Priority date	May 24, 2017
Publication date	Mar 31, 2026
Grant date	Mar 31, 2026

Abstract

Methods and apparatus improve a user experience during telephone calls or other forms of communication in which a listener localizes electronically generated binaural sounds. The sound is convolved or processed to a location that is behind or near a source of the sound so that the listener perceives the location of the sound as originating from the source of the sound.

First claim

What is claimed is:

1 . A method executed by a head mounted display (HMD) worn on a head of a user, the method comprising:
storing, in a memory of the HMD, head-related transfer functions (HRTFs) that are personalized to the user; scanning, with a camera in the HMD, an image of an iris of the user; determining, based on the image of the iris of the user, an identity of the user; retrieving, from the memory in the HMD and based on the identity of the user from the image of the iris, the HRTFs that are personalized to the user; and playing, through speakers in the HMD, binaural sound processed with the HRTFs that are personalized to the user; sensing, with one or more sensors in the HMD, when the user is walking or running; and lowering, by the HMD, a volume of the binaural sound playing through the speakers to the user in response to the HMD sensing the user is walking or running.

2 . The method of claim 1 further comprising:
joining, by the HMD, another virtual auditory space in response to the HMD sensing the user is walking or running.

3 . The method of claim 1 further comprising:
selecting, with the HMD, coordinates of the HRTFs that are on an imaginary line in a direction that coincides with a sound source at which the user is looking.

4 . The method of claim 1 further comprising:
capturing, with a camera in the HMD worn on the head of the user, real-time video of a person in a physical environment with the user;
displaying, with a display of the HMD and to the user, the real-time video of the person in the physical environment with the user; and
processing, with the HMD, the binaural sound with the HRTFs that are personalized to the user so the binaural sound originates from the real-time video of the person being displayed with the HMD.

5 . The method of claim 1 further comprising:
sensing, with the HMD, a person in a physical environment of the user; and
lowering a volume of the binaural sound playing through the speakers of the HMD in response to the HMD sensing the person in the physical environment of the user.

6 . A method executed by a head mounted display (HMD) worn on a head of a user, the method comprising:
storing, in memory of the HMD, head-related transfer functions (HRTFs) that are personalized to the user; determining, from images of a face of the user captured with a camera in the HMD while the user wears the HMD, an identity of the user; retrieving, from the memory, the HRTFs that are personalized to the user in response to determining the identity of the user wearing the HMD; and playing, with speakers in the HMD while the user wears the HMD, binaural sound processed with the HRTFs that are personalized to the user, wherein the binaural sound originates along a line-of-sight that occurs behind a sound source at which the user is looking.

7 . The method of claim 6 further comprising:
sensing, with the HMD, a person in a physical environment of the user; and
switching the binaural sound playing to the user to mono sound or stereo sound in response to the HMD sensing the person in the physical environment of the user.

8 . The method of claim 6 further comprising:
selecting, by the HMD, the HRTFs with coordinates that coincide with a line extending from the head of the user to a virtual image at which the user is looking.

9 . The method of claim 6 further comprising:
detecting, with the camera in the HMD, when the user is no longer looking at a sound source; and
pausing the playing of the binaural sound to the user in response to the camera detecting the user is no longer looking at the sound source.

10 . The method of claim 6 further comprising:
displaying, with a display of the HMD and to the user during a telephone call between the user and two people, a sound source as virtual images of the two people in front of the user; and
processing voices of the two people so the voices originate behind the virtual images of the two people in front of the user.

11 . The method of claim 6 further comprising:
sensing, with the HMD, a person in a physical environment of the user;
determining, with the HMD, a distance from the user to the person;
displaying, with the HMD, an image of the person; and
changing a volume of the binaural sound that plays to the user and originates from the image of the person as the distance from the user to the person changes.

12 . A head mounted display (HMD) worn on a head of a user, the HMD comprising:
a memory that stores head-related transfer functions (HRTFs) that are personalized to the user; speakers that play binaural sound processed with the HRTFs that are personalized to the user; a sensor that senses when the user is walking; and one or more sensors that scan an image of an iris of the user while the user wears the HMD, wherein the HMD determines, based on the image of the iris of the user, an identity of the user and retrieves, from the memory in the HMD and based on the identity of the user from the image of the iris, the HRTFs that are personalized to the user, and processes binaural sound with the HRTFs that are personalized to the user, and wherein the HMD lowers a volume of the binaural sound that plays to the user through the speakers in response to the sensor sensing the user is walking.

13 . The HMD of claim 12 further comprising:
a sensor that senses a person in a physical environment of the user while the user wears the HMD, wherein the HMD lowers a volume of the binaural sound playing to the user in response to sensing the person in the physical environment of the user while the user wears the HMD.

14 . The HMD of claim 12 , wherein the speakers pause playing of the binaural sound in response to sensing the user is no longer looking at a virtual image being displayed with the HMD.

Assignees

• Norris Glen A
• Lyren Philip Scott

Inventors

• Norris Glen A
• Lyren Philip Scott

Classifications

• H04N7/141

  between two video terminals, e.g. videophone (telephonic communication systems combined with television receiver for reception of entertainment or information matter H04M11/085) · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V40/165

  using facial parts and geometric relationships · CPC title