Wearable electronic device displays a 3D zone from where binaural sound emanates

Patent metadata

Field	Value
Publication number	US-11172316-B2
Application number	US-202017062633-A
Country	US
Kind code	B2
Filing date	Oct 5, 2020
Priority date	Feb 20, 2016
Publication date	Nov 9, 2021
Grant date	Nov 9, 2021

Abstract

A portable electronic device (PED) divides an area around a user into a three-dimensional (3D) zone. A wearable electronic device worn on a head of the user displays the 3D zone in response to the wearable electronic device detecting that the user is leaving the zone. The wearable electronic device plays binaural sound that emanates to the user from sound localization points (SLPs) inside the zone.

First claim

What is claimed is:

1. A method comprising:
dividing, with a portable electronic device (PED) held in a hand of a user, an area around the user into a zone that extends around the user and that includes a sound localization point (SLP) from where binaural sound in empty space originates to the user;
playing, with speakers in a wearable electronic device (WED) worn on a head of the user, the binaural sound that originates from the SLP in empty space;
tracking, with one or more sensors in the WED worn on the head of the user, the PED to determine when the PED held in the hand of the user is moving outside the zone that extends around the user; and
displaying, with the WED worn on the head of the user, a virtual reality (VR) image that shows the zone in response to the WED determining that the PED is moving outside the zone.

2. The method of claim 1 further comprising:
highlighting, while the user is located inside the zone and with a display of the WED, a VR image at the SLP in the zone when the PED held in the hand of the user is pointed at the SLP in the zone; and
playing, with the WED worn on the head of the user, the binaural sound that emanates from the SLP in response to the PED pointing at the SLP.

3. The method of claim 1 further comprising:
displaying, with the WED worn on the head of the user, the VR image that shows a size and a shape of the zone in response to the WED determining that the PED is moving outside the zone.

4. The method of claim 1 further comprising:
storing, in memory of the WED, a location of the zone, a size and a shape of the zone, and a location of the SLP in the zone from where the binaural sound originates to the user; and
retrieving, from the memory of the WED, the size and the shape of the zone in response to the WED determining that the user is in the location of the zone.

5. The method of claim 1 further comprising:
designating, with the PED held in the hand of the user, locations of multiple SLPs in empty space in the zone from where the binaural sound originates to the user; and
displaying, with the WED, VR images at the locations of the multiple SLPs in empty space in the zone.

6. The method of claim 1 further comprising:
selecting one of multiple SLPs in the zone when the PED held in the hand of the user is pointed at the one of the multiple SLPs in the zone; and
displaying, with the WED worn on the head of the user, a VR image at the one of the multiple SLPs in response to the one of the multiple SLPs being pointed at by the PED held in the hand of the user.

7. The method of claim 1 , wherein the zone has one of a three-dimensional (3D) shape of a cylinder and a 3D shape of a rectangle, and the user is centered in the zone.

8. The method of claim 1 further comprising:
playing, with the speakers in the WED worn on the head of the user, an audio warning in response to the WED determining that the PED is moving outside the zone.

9. A method comprising:
dividing, with a portable electronic device (PED) held in a hand of a user, an area around the user into a three-dimensional (3D) zone that extends around the user and that includes sound localization points (SLPs) from where binaural sound originates to the user;
determining, with a wearable electronic device (WED) worn on a head of the user, when the user is leaving the zone; and
warning the user that the user is leaving the zone by displaying, with the WED worn on the head of the user, a 3D virtual reality (VR) image of the zone in response to the WED determining that the user is leaving the zone.

10. The method of claim 9 further comprising:
determining, with the WED, when the PED is pointed at one of the SLPs while the user is located in the zone; and
playing the binaural sound that originates from the one of the SLPs in the zone in response to the WED determining that the PED is pointed at the one of the SLPs.

11. The method of claim 9 further comprising:
highlighting, while the user is located in the zone and with a display of the WED, color of a VR image at one of the SLPs in the zone when the PED held in the hand of the user points at the one of the SLPs in the zone.

12. The method of claim 9 , wherein the 3D VR image of the zone is shaped as a cylinder with the user being centered in the cylinder, and the WED is a head mounted display that display VR images and plays the binaural sound while the user plays a game.

13. The method of claim 9 , wherein the 3D VR image of the zone is shaped as a rectangle with the user being centered in the rectangle, and the WED is a head mounted display that displays VR images and plays the binaural sound while the user plays a game.

14. The method of claim 9 further comprising:
playing, with the WED worn on the head of the user, an audio warning indicating the user should move back into the zone in response to the WED determining that the user is leaving the zone.

15. The method of claim 9 , wherein the 3D VR image of the zone is shaped as a cylinder that extends around the user.

16. The method of claim 9 further comprising:
determining, with the WED worn on the head of the user, when the PED is leaving the zone; and
warning the user that the PED is leaving the zone by displaying, with the WED worn on the head of the user, the 3D VR image of the zone in response to the WED determining that the PED is leaving the zone.

17. An electronic system comprising:
a portable electronic device (PED) held in a hand of a user that divides an area around the user into a three-dimensional (3D) zone that extends around the user and includes sound localization points (SLPs) in empty space from where binaural sound originates to the user while the user is located inside the zone; and
a wearable electronic device (WED) that is worn on a head of the user, wirelessly communicates with the PED, includes speakers that play to the user the binaural sound that emanates from the SLPs in empty space inside the zone, includes one or more sensors that sense when the PED is leaving the zone, and includes a display that displays a virtual reality (VR) image of the zone in response to sensing that the PED is leaving the zone, wherein the display displays the VR image of the zone to warn the user that the user is leaving the zone.

18. The electronic system of claim 17 , wherein the display of the WED highlights a color of a VR image at one of the SLPs in response to the WED determining that the PED is pointing to the VR image at the one of the SLPs.

19. The electronic system of claim 17 , wherein the WED tracks a location of the user with respect to the SLPs, and automatically displays a VR image at a location of one of the SLPs in response to determining that the user is proximate to the location of the one of the SLPs.

20. The electronic system of claim 17 , wherein the speakers in the WED play the binaural sound that emanates from one of the SLPs in empty space in response to the WED determining that the PED is pointed to the one of the SLPs in empty space, and the zone is shaped as a 3D cylinder with the user being centered in the 3D cylinder.

Assignees

• Lyren Philip Scott
• Norris Glen A

Inventors

• Lyren Philip Scott
• Norris Glen A

Classifications

• G06V40/172

  Classification, e.g. identification · CPC title

• G06V40/168

  Feature extraction; Face representation · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• H04S7/304

  For headphones · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title