Designating a sound localization point (SLP) for binaural sound at a location of a handheld portable electronic device (HPED)

Patent metadata

Field	Value
Publication number	US-10602291-B2
Application number	US-201916445268-A
Country	US
Kind code	B2
Filing date	Jun 19, 2019
Priority date	Feb 20, 2016
Publication date	Mar 24, 2020
Grant date	Mar 24, 2020

Abstract

A wearable electronic device (WED) worn on a head of a user detects a location of a handheld portable electronic device (HPED) held in a hand of the user and designates a sound localization point (SLP) for binaural sound at the location of the HPED. A digital signal processor (DSP) processes sound to originate to the user as the binaural sound at the location after the HPED is removed from the location.

First claim

What is claimed is:

1. A method executed by one or more electronic devices in an electronic system, the method comprising:
detecting, with an electromagnetic motion tracker in a wearable electronic device (WED) worn on a head of a user, a location of a handheld portable electronic device (HPED) held in a hand of the user;
designating, by the WED worn on the head of the user, the location of the HPED as a sound localization point (SLP) from where binaural sound originates to the user in empty space after the HPED is moved from the location; and
processing, by a digital signal processor (DSP) and with at least one head-related transfer function (HRTF), sound into the binaural sound that originates to the user in empty space from the SLP at the location after the HPED is moved from the location.

2. The method of claim 1 further comprising:
wirelessly transmitting, from the HPED that is a smartphone to the WED that is electronic glasses, the binaural sound processed with the DSP that is in the smartphone.

3. The method of claim 1 further comprising:
providing, during a telephone call between the user and a person, the binaural sound to the user through headphones or earphones that communicate with the WED, wherein the sound is a voice of the person that originates to the user in empty space from the SLP at the location after the HPED is moved from the location.

4. The method of claim 1 further comprising:
storing, in the WED, a height of the HPED from ground, objects recognized in a room where the user is located, a room scan, and positions of the HPED and the user in the room.

5. The method of claim 1 further comprising:
displaying, with a display of the WED, the SLP as an augmented reality (AR) image or virtual reality (VR) image after the HPED is moved from the location.

6. The method of claim 1 further comprising:
designating a virtual speaker at the SLP at the location of the HPED; and
processing, with the DSP, the sound to originate in empty space at the SLP as the virtual speaker after the HPED is moved from the location.

7. The method of claim 1 further comprising:
simultaneously displaying, with a display of the WED, a plurality of SLPs;
selecting one of the plurality of SLPs that is being pointed to by the HPED; and
designating, from the selecting by the HPED, the one of the plurality of SLPs as a location from where the user hears the binaural sound.

8. The method of claim 1 further comprising:
simultaneously displaying, with a display of the WED, a plurality of SLPs with one of the SLPs being selected as a location where the binaural sound originates to the user; and
visually distinguishing, with the display of the WED, the one of the SLPs from non-selected ones of the plurality of SLPs to inform the user that the one of the SLPs is where the binaural sound originates to the user.

9. A non-transitory computer readable storage medium storing instructions that one or more electronic devices execute, the method comprising:
detecting, with a wearable electronic device (WED) worn on a head of a user, a location of a smartphone held in a hand of the user;
establishing the location of the smartphone as a sound localization point (SLP) from where binaural sound originates to the user in empty space when the smartphone is removed from the location; and
processing, by a digital signal processor (DSP) in the smartphone and with at least one head-related transfer function (HRTF), sound into the binaural sound that originates to the user in empty space from the SLP at the location after the smartphone is removed from the location.

10. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
capturing, with a camera in the smartphone, an image of a face of the user;
determining, from the image, an azimuth coordinate of the smartphone with respect to a head orientation of the user; and
designating the SLP in empty space to have the azimuth coordinate of the smartphone.

11. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
displaying, with the WED worn on the head of the user, an augmented reality (AR) image at the location after the smartphone is removed from the location, wherein the AR image that is displayed during a telephone call as a person in the telephone call.

12. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
receiving, at one of the WED or the smartphone, a voice command from the user that establishes the SLP.

13. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
automatically establishing the SLP when the smartphone is at a predetermined distance from the head of the user.

14. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
generating, with the smartphone, a plurality of SLPs while the smartphone moves to different locations with respect to the head of the user;
mapping, with the smartphone, locations of the plurality of SLPs with respect to the head of the person; and
displaying, on a display of the WED, the locations of the plurality of SLPs in a three-dimensional (3D) map.

15. A method executed by one or more electronic devices in an electronic system, the method comprising:
detecting, with a wearable electronic device (WED) worn on a head of a user, a location of a handheld portable electronic device (HPED) held in a hand of the user;
designating the location of the HPED as a sound localization point (SLP) from where binaural sound originates to the user in empty space when the HPED is removed from the location; and
processing, by a digital signal processor (DSP) and with at least one head-related transfer function (HRTF), sound into the binaural sound that originates to the user in empty space from the SLP at the location when the HPED is removed from the location.

16. The method of claim 15 further comprising:
scanning, with the one or more electronic devices in the electronic system, for SLPs that exist in an environment of the user;
displaying, with the WED, the SLPs to the user; and
receiving, from the user and at the one or more electronic devices in the electronic system, a command to select one of the SLPs as the SLP where binaural sound externally localizes to the user during a telephone call.

17. The method of claim 15 further comprising:
dividing, by the one or more electronic devices in the electronic system, an area around the head of the user into a plurality of three-dimensional (3D) zones;
determining, by the one or more electronic devices in the electronic system, in which one of the zones the HPED is located; and
capturing, by the one or more electronic devices in the electronic system, coordinates of the SLP when the HPED is located in the one of the zones.

18. The method of claim 15 further comprising:
calculating, by the one or more electronic devices in the electronic system, one or more coordinates of the location of the HPED with respect to a face of the user located at an origin in a coordinate system; and
saving, in a memory of the one or more electronic devices in the electronic system, the one or more coordinates of the location of the HPED as coordinates of the SLP in the coordinate system.

19. The method of claim 15 further comprising:
receiving, at the HPED and when the HPED is one meter away from the head of the user, a command from the person to generate the SLP wh

Assignees

• Lyren Philip Scott
• Norris Glen A

Inventors

• Lyren Philip Scott
• Norris Glen A

Classifications

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• H04S7/301

  Automatic calibration of stereophonic sound system, e.g. with test microphone · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• H04R29/00

  Monitoring arrangements; Testing arrangements {(for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308)} · CPC title

• H04R2499/11

  Transducers incorporated or for use in hand-held devices, e.g. mobile phones, PDA's, camera's · CPC title