Providing binaural sound behind a virtual image being displayed with a wearable electronic device (WED)

Patent metadata

Field	Value
Publication number	US-11889289-B2
Application number	US-202217703528-A
Country	US
Kind code	B2
Filing date	Mar 24, 2022
Priority date	May 24, 2017
Publication date	Jan 30, 2024
Grant date	Jan 30, 2024

Abstract

A wearable electronic device (WED) worn on a head of a user displays a virtual image that has sound. One or more processors process the sound for the virtual image into binaural sound for the user. The binaural sound has a sound localization point (SLP) with a coordinate location that occurs behind the virtual image while the image is located a near-field distance from the head of the user.

First claim

What is claimed is:

1. A method, comprising:
displaying, with a wearable electronic device (WED) worn on a head a user, a virtual object that appears within a near-field distance from the user; and
processing, with one or more processors in the WED, sound for the virtual object to generate binaural sound that has a sound localization point (SLP) with a coordinate location at a far-field distance from the user that occurs along a line-of-sight from the user to the virtual object and behind the virtual object while the virtual object is within the near-field distance from the user.

2. The method of claim 1 further comprising:
tracking, with one or more sensors in the WED, movement of the head of the user; and
processing, with the one or more processors and while the head of the user moves with respect to the virtual object, the sound for the virtual object such that the SLP remains behind the virtual object while the virtual object remains within the near-field distance.

3. The method of claim 1 further comprising:
continuing to process, with the one or more processors, the sound for the virtual object with far-field head-related transfer functions (HRTFs) while the sound emanates from the virtual object with the virtual object located within the near-field distance from the user.

4. The method of claim 1 further comprising:
sensing, with one or more sensors in the WED, movement of the WED while worn on the head of the user; and
automatically lowering a volume of the sound in response to the one or more sensors sensing the movement.

5. The method of claim 1 , wherein the one or more processors include a digital signal processor (DSP) that processes the sound for the virtual object that is displayed within the near-field distance from the user with far-field head-related transfer functions (HRTFs) as long as the sound originates from the virtual object with the user located within the near-field distance from the virtual object.

6. The method of claim 1 , wherein the coordinate location of the SLP is located a distance (d1) from the head of the user in which d1>1.0 meter, the virtual object is located a distance (d2) from the head of the user in which d2<1.0 meter, the WED is a head mounted display (HMD), and the virtual object occurs in virtual reality (VR).

7. The method of claim 1 further comprising:
sensing, with one or more sensors in the WED, movement of the WED while worn on the head of the user; and
automatically muting the sound in response to the one or more sensors sensing the movement.

8. A wearable electronic device (WED) worn on a head of a user, the WED comprising:
a display that displays a virtual image that occurs within a near-field distance from the head of the user; and
a digital signal processor (DSP) that processes sound for the virtual image with far-field head-related transfer functions (HRTFs) to generate binaural sound that has a sound localization point (SLP) that occurs at a far-field distance located behind the virtual image and along a line-of-sight from the user to the virtual image while the virtual image is located within a near-field distance from the head of the user.

9. The WED of claim 8 , wherein the virtual image is an augmented reality (AR) image, and the display is included with electronic glasses worn on the head of the user.

10. The WED of claim 8 , wherein the virtual image is a virtual reality (VR) image, and the display is included with a head mounted display (HMD) worn on the head of the user.

11. The WED of claim 8 further comprising:
one or more sensors that sense movement of the WED while the WED is worn on the head of the user, wherein the WED lowers a volume of the sound in response to the one or more sensors sensing the movement.

12. The WED of claim 8 further comprising:
one or more sensors that sense movement of the WED while the WED is worn on the head of the user, wherein the WED mutes the sound in response to the one or more sensors sensing the movement.

13. The WED of claim 8 further comprising:
one or more sensors that sense movement of the WED while the WED is worn on the head of the user, wherein the WED terminates an electronic call in response to the one or more sensors sensing the movement.

14. The WED of claim 8 , wherein the virtual image is an augmented reality (AR) image of an intelligent user assistant (IUA) that is located less than one meter away from the head of the user, and a voice of the IUA occurs at a coordinate location behind the AR image of the IUA when the AR image is within the near-field distance from the head of the user.

15. The WED of claim 8 , wherein the virtual image is a virtual reality (VR) image of an intelligent user assistant (IUA) that is located less than one meter away from the head of the user, and a voice of the IUA occurs at a coordinate location behind the VR image of the IUA when the VR image is within the near-field distance from the head of the user.

16. A wearable electronic device (WED) worn on a head of a user, the WED comprising:
a display that displays a virtual image that is located within a near-field distance from the head of the user viewing the virtual image; and
one or more processors that process sound for the virtual image with head-related transfer functions (HRTFs) to generate binaural sound that has a sound localization point (SLP) with a coordinate location that occurs at a far-field distance located behind the virtual image along a line-of-sight from the user to the virtual image while the display displays the virtual image to the user at the near-field distance, wherein the coordinate location of the binaural sound continues to occur at the far-field distance located behind the virtual image when the head of the user moves while the virtual image remains within the near-field distance from the head of the user.

17. The WED of claim 16 , wherein the display is an augmented reality (AR) display worn on the head of the user viewing the virtual image, and the virtual image is an AR image.

18. The WED of claim 16 , wherein the display is a virtual reality (VR) display worn on the head of the user viewing the virtual image, and the virtual image is a VR image.

19. The WED of claim 16 further comprising:
one or more sensors that sense a direction of gaze of the user, wherein the WED mutes the sound in response to sensing that the direction of gaze is not directed to the virtual image.

20. The WED of claim 16 further comprising:
one or more sensors that sense a direction of gaze of the user, wherein the WED switches the sound to mono sound or stereo sound in response to sensing that the direction of gaze is not directed to the virtual image.

Assignees

• Norris Glen A
• Lyren Philip Scott

Inventors

• Norris Glen A
• Lyren Philip Scott

Classifications

• H04S7/303 Primary

  Tracking of listener position or orientation · CPC title

• G06V10/17

  using hand-held instruments · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G06V40/165

  using facial parts and geometric relationships · CPC title

• G06V40/172

  Classification, e.g. identification · CPC title