Camera driven audio spatialization

Patent metadata

Field	Value
Publication number	US-9596555-B2
Application number	US-201213628464-A
Country	US
Kind code	B2
Filing date	Sep 27, 2012
Priority date	Sep 27, 2012
Publication date	Mar 14, 2017
Grant date	Mar 14, 2017

Abstract

A camera input can be used by the computer to support audio spatialization or to improve audio spatialization of an application that already supports it. A computer system may to support audio spatialization, for example, by modifying the relative latency or relative amplitude of the rendered audio packets. If a sound is intended, for example, to be located on the left side of the user, then the audio channel that is rendered on the headset speaker located on the user's left ear may have a somewhat decreased latency and increased amplitude compared to the other audio channel.

First claim

What is claimed is:

1. A method comprising:
using video analytics to determine the position and orientation of a user's head relative to a processor based system;
based on said position and orientation, adjusting sound provided by said processor based system to headphones to produce three-dimensional sound effects which depends on distance of the user from a display of the processor based system and the orientation of the user's head relative to the display, wherein,
adjusting the sound includes adjusting latency of audio streams produced by said processor based system, based on the distance of the user from the display and the orientation of the user's head relative to the display.

2. The method of claim 1 including adjusting the amplitude of audio streams produced by said processor based system based on said video analytics.

3. The method of claim 1 including using a digital signal processor to produce said three-dimensional effects.

4. The method of claim 1 including introducing an appropriate delay or amplitude manipulation on each of two audio streams to spatialize them based on the position and orientation of the user's head.

5. The method claim 1 including periodically determining the position of the user's head.

6. The method claim 1 including periodically determine the orientation of the user's head.

7. One or more non-transitory computer readable media storing instructions executed by a computer to perform a sequence comprising:
using video analytics to determine the position and orientation, of a user's head relative to a processor based system;
based on said position and orientation, adjusting sound provided by said processor based system to headphones to produce three-dimensional sound effects which depends on distance of the user from a display of the processor based system and the orientation of the user's head relative to the display, wherein,
adjusting the sound includes adjusting latency of audio streams produced by said processor based system, based on the distance of the user from the display and the orientation of the user's head relative to the display.

8. The media of claim 7 , the sequence including adjusting the amplitude of audio streams produced by said processor based system based on said video analytics.

9. The media of claim 7 , the sequence including using a digital signal processor to produce said three-dimensional effects.

10. The media of claim 7 , the sequence including introducing an appropriate delay or amplitude manipulation on each of two audio streams to spatialize them based on the position and orientation of the user's head.

11. The media of claim 7 , the sequence including periodically determining the position of the user's head.

12. The media of claim 7 , the sequence including periodically determine the orientation of the user's head.

13. An apparatus comprising:
a processor to adjust sound based on a position and orientation, of a user by adjusting latency of audio streams produced by said processor based system to headphones and the orientation of the user's head relative to a display, wherein; and to adjust to the sound includes adjusting the latency of audio streams produced by said processor based system, based on a distance of the user from the display and the orientation of the user's head relative to the display; and
an audio codec coupled to said processor.

14. The apparatus of claim 13 wherein said processor is a digital signal processor.

15. The apparatus of claim 13 including a device to use video analytics to determine the user's position.

16. The apparatus of claim 15 wherein said device includes said processor.

17. The apparatus of claim 16 wherein said device includes a camera.

18. The apparatus of claim 13 said processor to determine an orientation of the user's head.

19. The apparatus of claim 13 said processor to adjust for the distance of the user from the computer for the processor based system.

20. The apparatus of claim 19 said processor to adjust for the orientation of the user's head relative to the computer of the processor based system.

21. The apparatus of claim 13 including an operating system.

22. The apparatus of claim 13 including a battery.

23. The apparatus of claim 13 including firmware and a module to update said firmware.

Assignees

• Intel Corp

Inventors

• Kaburlasos Nikos
• Cheng Scott W
• Worrell Devon

Classifications

• H04S2420/01

  Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] · CPC title

• H04R5/02 Primary

  Spatial or constructional arrangements of loudspeakers · CPC title

• H04S5/00 Primary

  Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation · CPC title

• H04S7/304 Primary

  For headphones · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title