Method, apparatus and computer program for generating an spatial audio output based on an spatial audio input

Patent metadata

Field	Value
Publication number	US-9591418-B2
Application number	US-201314391851-A
Country	US
Kind code	B2
Filing date	Feb 21, 2013
Priority date	Apr 13, 2012
Publication date	Mar 7, 2017
Grant date	Mar 7, 2017

Abstract

A method, apparatus and computer program for: receiving a spatial audio input; determining a direction of interest from the spatial audio input; and generating a spatial audio output dependent on the spatial audio input and the direction of interest.

First claim

We claim:

1. A computer implemented method comprising causing, at least in part, actions that result in:
receiving, at at least one processor, a spatial audio input;
determining, by the at least one processor, a direction of interest from the spatial audio input;
receiving, at the at least one processor, a spatial visual input;
determining, by the at least one processor, positional information related to one or more imaged objects from the spatial visual input;
determining, by the at least one processor, if one imaged object is eclipsed by another imaged object based on the positional information; and
generating, by the at least one processor, a spatial audio output based on the spatial audio input, wherein the spatial audio output generated is dependent on:
the direction of interest,
the positional information, and
the determination if one imaged object is eclipsed by another imaged object.

2. The method of claim 1 , wherein the positional information relates to one or more perceived depths of the one or more imaged object.

3. The method of claim 1 , further comprising identifying, by the at least processor, an aural cue from the spatial audio input, and preferably further comprising determining, from the spatial audio input, positional information of the aural cue, and yet further preferably wherein the determination of the direction of interest is dependent on the positional information of the aural cue.

4. The method of claim 1 , wherein the spatial audio output comprises an aural spatial directivity and the direction of the aural spatial directivity is aligned with the direction of interest.

5. The method of claim 1 , further comprising generating, by the at least one processor, a visual output dependent on the direction of interest.

6. The method of claim 5 , wherein the visual output comprises a visual spatial directivity and the direction of the visual spatial directivity is aligned with the direction of interest.

7. The method of claim 5 , wherein the visual output is associated with a position of a field of view and wherein the determination of the direction of interest is dependent on the position of the field of view.

8. The method of claim 5 , wherein the visual output is associated with a direction of a field of view, and wherein the direction of the field of view is dependent on the direction of interest.

9. The method of claim 5 , wherein the spatial visual input is configured to represent information relating to visual spatial directivity, and wherein the generation of the visual output comprises at least one of:
selecting a direction of visual spatial directivity of the visual output that corresponds to the direction of interest, and
synthesising, from the spatial visual input, an image associated with a field of view having a direction that corresponds to the direction of interest.

10. The method of claim 5 , further comprising:
determining, at the at least one processor, from the visual output, positional information of an imaged object; and
wherein the generating of a spatial audio output is further dependent on the positional information.

11. The method of claim 10 , wherein the spatial audio input comprises positional information relating to an audio source, the method further comprising associating the imaged object with the audio source.

12. The method of claim 5 , further comprising:
determining, by the at least one processor, a first set of positional information relating to a plurality of imaged objects at a first field of view based on the visual output;
determining, by the at least one processor, a second set of positional information relating to the plurality of imaged objects at a second field of view based on the visual output; and
wherein the generating of a spatial audio output is further dependent on the first and second sets of positional information.

13. The method of claim 12 , further comprising:
determining, by the at least one processor, if one of the plurality of imaged objects is eclipsed by another of the plurality of imaged objects when changing from the first field of view to the second field of view.

14. An apparatus comprising:
at least one processor; and
at least one memory storing computer program instructions configured, working with the at least one processor, to cause the apparatus to perform:
receiving a spatial audio input;
determining a direction of interest from the spatial audio input;
receiving a spatial visual input;
determining positional information related to one or more imaged objects from the spatial visual input;
determining, by the at least one processor, if one imaged object is eclipsed by
another imaged object based on the positional information: and
generating a spatial audio output based on the spatial audio input, wherein the spatial audio output generated is dependent on:
the direction of interest,
the positional information, and
the determination if one imaged object is eclipsed by another imaged object.

15. The apparatus of claim 14 , wherein the positional information relates to one or more perceived depths of the one or more imaged objects.

16. A module or a device comprising the apparatus of claim 14 .

17. A non-transitory computer readable medium embodying computer program instructions that, when performed by at least one processor, causes method as claimed in claim 1 to be performed.

18. The method of claim 1 , wherein the spatial audio output is generated from the spatial audio input by adjusting a spatial aural directivity of the spatial audio input.

Assignees

• Nokia Corp
• Nokia Technologies Oy

Inventors

• Shenoy Ravi
• Ukil Soumik
• Putraya Gururaj
• Govindarao Krishna

Classifications

• H04R29/00 Primary

  Monitoring arrangements; Testing arrangements {(for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308)} · CPC title

• H04S7/302 Primary

  Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence) · CPC title

• G06F3/16

  Sound input; Sound output (speech processing G10L) · CPC title