Spatial audio navigation

Patent metadata

Field	Value
Publication number	US-12188777-B2
Application number	US-202318338193-A
Country	US
Kind code	B2
Filing date	Jun 20, 2023
Priority date	Sep 27, 2017
Publication date	Jan 7, 2025
Grant date	Jan 7, 2025

Abstract

Methods and apparatus for spatial audio navigation that may, for example, be implemented by mobile multipurpose devices. A spatial audio navigation system provides navigational information in audio form to direct users to target locations. The system uses directionality of audio played through a binaural audio device to provide navigational cues to the user. A current location, target location, and map information may be input to pathfinding algorithms to determine a real world path between the user's current location and the target location. The system may then use directional audio played through a headset to guide the user on the path from the current location to the target location. The system may implement one or more of several different spatial audio navigation methods to direct a user when following a path using spatial audio-based cues.

First claim

What is claimed is:

1. A system for guiding a user along a path, the system comprising:
a mobile device, comprising:
one or more processors;
one or more position sensors to detect and track a current location of the mobile device; and
a memory comprising program instructions executable by the one or more processors to:
determine a target location of the mobile device;
determine a path from the current location to the target location, wherein the path includes at least one turn;
control virtual directionality and distance of sound played through a binaural audio device based on the current location of the mobile device relative to the path;
detect that a turn in the path has been missed; and
in response to said detection that a turn in the path has been missed, adjust the virtual directionality of the sound so that the sound appears to come from the direction of the missed turn as an indication that the turn has been missed.

2. The system as recited in claim 1 , wherein in response to said detection that a turn in the path has been missed, the program instructions are further executable by the one or more processors to attenuate one or more of volume, frequency, and reverberation of the sound as a further indication that the turn has been missed.

3. The system as recited in claim 1 , wherein in response to said detection that a turn in the path has been missed, the program instructions are further executable by the one or more processors to play an alert sound as a further indication that the turn has been missed.

4. The system as recited in claim 1 , wherein in response to said detection that a turn in the path has been missed, the program instructions are further executable by the one or more processors to determine a new path from the current location of the mobile device to the target location.

5. The system as recited in claim 1 , wherein the system further includes the binaural audio device, wherein the binaural audio device is connected to the mobile device by a wired or wireless connection, and wherein the program instructions are further executable by the one or more processors to detect that the turn in the path has been missed and adjust the virtual directionality of the sound played through the binaural audio device based at least in part on data indicative of a motion and orientation of the binaural audio device.

6. The system as recited in claim 5 , wherein the system further includes the binaural audio device, wherein the binaural audio device includes left and right audio output components, and wherein the virtual directionality and distance of the sound played through the binaural audio device are controlled by adjusting one or more of volume, frequency, or reverberation of left and right audio channels transmitted to the left and right audio output components of the binaural audio device by the mobile device over the wired or wireless connection.

7. The system as recited in claim 1 , wherein the sound played through the binaural audio device includes one or more of music, a telephone conversation, an audio book, a podcast, a radio broadcast, ambient noise sampled from or simulated for a real environment, simulated sounds including tones and white noise, or recorded sounds.

8. The system as recited in claim 1 , wherein, to control virtual directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path, the program instructions are further executable by the one or more processors to apply a spatial audio navigation method in which the sound is moved continuously along the path.

9. The system as recited in claim 1 , wherein, to control virtual directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path, the program instructions are further executable by the one or more processors to apply a spatial audio navigation method in which the sound is moved in discrete intervals along the path.

10. The system as recited in claim 1 , wherein, to control virtual directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path, the program instructions are further executable by the one or more processors to apply a spatial audio navigation method in which the sound is positioned at a distance from the current location of the binaural audio device in a direction along the path.

11. The system as recited in claim 1 , wherein, to control virtual directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path, the program instructions are further executable by the one or more processors to apply a spatial audio navigation method in which the sound moves from a position behind the current location of the binaural audio device on the path to a position in front of the current location of the binaural audio device on the path.

12. The system as recited in claim 1 , wherein, to control virtual directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path, the program instructions are further executable by the one or more processors to apply a spatial audio navigation method in which sampled or simulated ambient noise is played in a direction along the path and blocked in other directions to cause a sound tunneling effect.

13. The system as recited in claim 1 , wherein, to control virtual directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path, the program instructions are further executable by the one or more processors to apply a spatial audio navigation method in which sampled or simulated ambient noise is blocked in a direction along the path and played in other directions to cause a gap in the ambient noise.

14. The system as recited in claim 1 , wherein, to control virtual directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path, the program instructions are further executable by the one or more processors to apply a spatial audio navigation method in which the sound is played in a direction opposite that of the path.

15. The system as recited in claim 1 , wherein the system further includes the binaural audio device, wherein the binaural audio device is a headset, and wherein the mobile device is a mobile multipurpose device.

16. The system as recited in claim 1 , wherein the system further includes the binaural audio device, wherein the mobile device is a head-mounted display device, and wherein the binaural audio device is one of a headset or speakers integrated in the head-mounted display device.

17. A method, comprising:
performing, by a mobile device comprising one or more processors:
determining a current location and a target location;
determining a path from the current location to the target location, wherein the path includes at least one turn;
adjusting one or more of volume, frequency, or reverberation of left and right audio channels transmitted to left and right audio output components of a binaural audio device to control directionality and distance of sound played through the binaural audio device based on the current location of the mobile device relative to the path; and
in response to detecting that a turn in the path has been missed, adjusting vir

Assignees

• Apple Inc

Inventors

• Sommer Bruno M
• Bar-Zeev Avi
• Angermann Frank
• Pinto Stephen E
• Jonsson Lilli Ing-Marie
• Nair Rahul

Classifications

• H04S2400/13

  Aspects of volume control, not necessarily automatic, in stereophonic sound systems · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S7/304

  For headphones · CPC title

• H04S1/007

  in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02) · CPC title

• H04R2499/15

  Transducers incorporated in visual displaying devices, e.g. televisions, computer displays, laptops · CPC title