Wearable electronic device displays a 3D zone from where binaural sound emanates

Patent metadata

Field	Value
Publication number	US-10798509-B1
Application number	US-202016938936-A
Country	US
Kind code	B1
Filing date	Jul 25, 2020
Priority date	Feb 20, 2016
Publication date	Oct 6, 2020
Grant date	Oct 6, 2020

Abstract

A portable electronic device (PED) divides an area around a user into a three-dimensional (3D) zone. A wearable electronic device worn on a head of the user displays the 3D zone in response to the wearable electronic device detecting that the user is leaving the zone. The wearable electronic device plays binaural sound that emanates to the user from sound localization points (SLPs) inside the zone.

First claim

What is claimed is:

1. A method comprising:
dividing, with a portable electronic device (PED) held in a hand of a user, an area around the user into a zone that includes multiples sound localization points (SLPs) from where binaural sound originates to the user;
determining, with a wearable electronic device (WED) worn on a head of the user, when the user is located inside the zone;
highlighting, while the user is located in the zone and with a display of the WED, one of the multiple SLPs in the zone when the PED held in the hand of the user is pointed at the one of the multiple SLPs in the zone; and
playing, with the WED worn on the head of the user, the binaural sound that emanates from the one of the multiple SLPs in response to the one of the multiple SLPs being pointed at by the PED held in the hand of the user.

2. The method of claim 1 further comprising:
tracking, with one or more sensors in the WED worn on the head of the user, the PED to determine when the PED moves outside the zone; and
displaying, with the WED worn on the head of the user, a three dimensional (3D) virtual image that shows a size and a shape of the zone in response to the WED determining that the PED moved outside the zone.

3. The method of claim 1 further comprising:
tracking, with the WED worn on the head of the user, the user to determine when the user moves outside the zone; and
displaying, with the WED worn on the head of the user, a three dimensional (3D) virtual image that shows a size and a shape of the zone in response to the WED determining that the user moved outside the zone.

4. The method of claim 1 further comprising:
storing, in memory of the WED, a location of the zone, a size and a shape of the zone, and locations of the multiple SLPs in the zone from where the binaural sound originates to the user; and
displaying, with the WED, a virtual image having the size and the shape of the zone in response to the WED determining that the user is located in the zone.

5. The method of claim 1 further comprising:
designating, with the PED held in the hand of the user, locations of the multiple SLPs in the zone from where the binaural sound originates to the user; and
displaying, with the WED, virtual images at the location of the multiple SLPs in the zone.

6. The method of claim 1 further comprising:
selecting the one of the multiple SLPs in the zone when the PED held in the hand of the user is pointed at the one of the multiple SLPs in the zone; and
displaying, with the WED worn on the head of the user, a virtual image at the one of the multiple SLPs in response to the one of the multiple SLPs being pointed at by the PED held in the hand of the user.

7. The method of claim 1 , wherein the zone has one of a three-dimensional (3D) shape of a cylinder and a 3D shape of a rectangle, and the user is centered in the zone.

8. The method of claim 1 further comprising:
playing, with speakers in the WED worn on the head of the user, a verbal instruction indicating the user should move back into the zone in response to the WED determining that the user moved outside the zone.

9. A non-transitory computer readable storage medium storing instructions that one or more electronic devices execute as a method, the method comprising:
dividing, with a portable electronic device (PED) held in a hand of a user, an area around the user into a zone that includes sound localization points (SLPs) in empty space from where binaural sound originates to the user;
determining, with a wearable electronic device (WED) worn on a head of the user, when the user is located inside the zone;
determining, with the WED worn on the head of the user, when the user is leaving the zone; and
displaying, with the WED worn on the head of the user, a three dimensional (3D) virtual image of the zone in response to the WED determining that the user is leaving the zone.

10. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
determining, with the WED, when the PED is pointed at one of the SLPs while the user is located in the zone; and
playing the binaural sound that originates from the one of the SLPs in the zone in response to the WED determining that the PED is pointed at the one of the SLPs.

11. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
highlighting, while the user is located in the zone and with a display of the WED, a virtual image at one of the SLPs in the zone when the PED held in the hand of the user is pointed at the one of the multiple SLPs in the zone.

12. The non-transitory computer readable storage medium of claim 9 , wherein the 3D virtual image of the zone is shaped as a cylinder with the user being centered in the cylinder, and the WED is one of a head mounted display and electronic glasses.

13. The non-transitory computer readable storage medium of claim 9 , wherein the 3D virtual image of the zone is shaped as a rectangle with the user being centered in the rectangle, and the WED is one of a head mounted display and electronic glasses.

14. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
playing, with the WED worn on the head of the user, a verbal instruction indicating the user should move back into the zone in response to the WED determining that the user moved outside the zone.

15. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
designating, with the PED held in the hand of the user, a location on a physical object pointed to by the PED as a virtual right speaker and a location on the physical object pointed to by the PED as a virtual left speaker; and
playing, with the WED worn on the head of the user, the binaural sound that emanates from the virtual right speaker and the virtual left speaker.

16. The non-transitory computer readable storage medium of claim 9 in which the method further comprises:
determining room impulse responses (RIRs) based on a size and a shape of the zone;
storing the RIRs in memory of the WED; and
retrieving, from the memory of the WED, the RIRs in response to the WED entering the zone.

17. An electronic system comprising:
a portable electronic device (PED) held in a hand of a user that divides an area around the user into a three-dimensional (3D) zone that includes sound localization points (SLPs) in empty space from where binaural sound originates to the user; and
a wearable electronic device (WED) that is worn on a head of the user, wirelessly communicates with the PED, includes speakers that play the binaural sound to the user that emanates from the SLPs in empty space, includes one or more sensors that sense when the PED leaves the zone, and includes a display that displays a 3D virtual image of the zone in response to sensing that the PED leaves the zone, wherein the display displays the 3D virtual image of the zone to warn the user that the user is leaving the zone.

18. The electronic system of claim 17 , wherein the display of the WED highlights a virtual image at one of the SLPs in response to the WED determining that the PED is pointing to the one of the SLPs.

19. The electronic system of claim 17 , wherein the WED tracks a location of the user with respect to the SLPs, and automatically display a virtual image at a location of one of the SLPs in response to determining that the user is proximate to the location of the one of the SLPs.

20. The electronic syst

Assignees

• Lyren Philip Scott
• Norris Glen A

Inventors

• Lyren Philip Scott
• Norris Glen A

Classifications

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V40/168

  Feature extraction; Face representation · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title