Dynamic speech directivity reproduction

Patent metadata

Field	Value
Publication number	US-11096006-B1
Application number	US-201916672549-A
Country	US
Kind code	B1
Filing date	Nov 4, 2019
Priority date	Nov 4, 2019
Publication date	Aug 17, 2021
Grant date	Aug 17, 2021

Abstract

The disclosed computer-implemented method may include capturing, via a headset microphone of a speaker's artificial reality device, voice input of a speaker in communication with a listener in an artificial reality environment. The method may include detecting a pose of the speaker within the artificial reality environment and determining a position of the speaker relative to a position of the listener within the artificial reality environment. The method may further include processing, based on the pose and the relative position of the speaker within the artificial reality environment, the voice input to create a directivity-attuned voice signal for the listener, and delivering the directivity-attuned voice signal to an artificial reality device of the listener. Various other methods, systems, and computer-readable media are also disclosed.

First claim

What is claimed is:

1. A method comprising:
capturing, via a headset microphone of a speaker's artificial reality device, voice input of a speaker in communication with a listener in an artificial reality environment;
detecting a pose of the speaker within the artificial reality environment;
determining a position of the speaker relative to a position of the listener within the artificial reality environment;
determining a directivity profile for the speaker;
determining a directivity pattern for the voice input based on the pose, the relative position of the speaker within the artificial reality environment, and the directivity profile;
processing, using the directivity pattern, the voice input to create a directivity-attuned voice signal for the listener; and
delivering the directivity-attuned voice signal to an artificial reality device of the listener.

2. The method of claim 1 , wherein the directivity profile is determined based on a content of the voice input such that the directivity-attuned voice signal is created in a manner that accounts for the content of the voice input.

3. The method of claim 1 , wherein the directivity profile is determined based on at least one of a physical characteristic of the speaker, a voice frequency range of the speaker, or a headset size of the speaker such that the directivity-attuned voice signal is created in a manner that accounts for the physical characteristic of the speaker, the voice frequency range of the speaker, or the headset size of the speaker.

4. The method of claim 1 , wherein the directivity profile is determined based on a gender of the speaker such that the directivity-attuned voice signal is created in a manner that accounts for the gender of the speaker.

5. The method of claim 1 , wherein creating the directivity-attuned voice signal further comprises:
identifying, in the voice input, reverberation from a real-world environment of the speaker; and
removing, from the voice input, at least a portion of the reverberation.

6. The method of claim 1 , wherein creating the directivity-attuned voice signal further comprises:
identifying a reverberant property of an artificial reality environment of the listener; and
adding, to the voice input, reverberation based on the reverberant property of the artificial reality environment of the listener.

7. A system comprising:
at least one physical processor;
physical memory comprising computer-executable instructions that, when executed by the physical processor, cause the physical processor to:
capture, via a headset microphone of a speaker's artificial reality device, voice input of a speaker in communication with a listener in an artificial reality environment;
detect a pose of the speaker within the artificial reality environment;
determine a position of the speaker relative to a position of the listener within the artificial reality environment;
determine a directivity profile for the speaker;
determine a directivity pattern for the voice input based on the pose, the relative position of the speaker within the artificial reality environment, and the directivity profile;
process, using the directivity pattern, the voice input to create a directivity-attuned voice signal for the listener; and
deliver the directivity-attuned voice signal to an artificial reality device of the listener.

8. The system of claim 7 , wherein the directivity profile is determined based on a content of the voice input such that the directivity-attuned voice signal is created in a manner that accounts for the content of the voice input.

9. The system of claim 7 , wherein the directivity profile is determined based on at least one of a physical characteristic of the speaker, a voice frequency range of the speaker, or a headset size of the speaker such that the directivity-attuned voice signal is created in a manner that accounts for the physical characteristic of the speaker, the voice frequency range of the speaker, or the headset size of the speaker.

10. The system of claim 7 , wherein the directivity profile is determined based on a gender of the speaker such that the directivity-attuned voice signal is created in a manner that accounts for the gender of the speaker.

11. The system of claim 7 , wherein creating the directivity-attuned voice signal further comprises:
identifying, in the voice input, reverberation from a real-world environment of the speaker; and
removing, from the voice input, at least a portion of the reverberation.

12. The system of claim 7 , wherein creating the directivity-attuned voice signal further comprises:
identifying a reverberant property of an artificial reality environment of the listener; and
adding, to the voice input, reverberation based on the reverberant property of the artificial reality environment of the listener.

13. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to:
capture, via a headset microphone of a speaker's artificial reality device, voice input of a speaker in communication with a listener in an artificial reality environment;
detect a pose of the speaker within the artificial reality environment;
determine a position of the speaker relative to a position of the listener within the artificial reality environment;
determine a directivity profile for the speaker;
determine a directivity pattern for the voice input based on the pose, the relative position of the speaker within the artificial reality environment, and the directivity profile;
process, using the directivity pattern, the voice input to create a directivity-attuned voice signal for the listener; and
deliver the directivity-attuned voice signal to an artificial reality device of the listener.

14. The computer-readable medium of claim 13 , wherein the directivity profile is determined based on a content of the voice input such that the directivity-attuned voice signal is created in a manner that accounts for the content of the voice input.

15. The computer-readable medium of claim 13 , wherein the directivity profile is determined based on at least one of a gender of the speaker, a physical characteristic of the speaker, a voice frequency range of the speaker, or a headset size of the speaker such that the directivity-attuned voice signal is created in a manner that accounts for the gender of the speaker, the physical characteristic of the speaker, the voice frequency range of the speaker, or the headset size of the speaker.

16. The computer-readable medium of claim 13 , wherein creating the directivity-attuned voice signal further comprises:
identifying, in the voice input, reverberation from a real-world environment of the speaker; and
removing, from the voice input, at least a portion of the reverberation.

17. The computer-readable medium of claim 13 , wherein creating the directivity-attuned voice signal further comprises:
identifying a reverberant property of an artificial reality environment of the listener; and
adding, to the voice input, reverberation based on the reverberant property of the artificial reality environment of the listener.

18. The method of claim 1 , wherein the directivity-attuned voice signal reproduces a dynamic speech directivity of the speaker within the artificial reality environment.

19. The system of claim 7 , wherein the directivity-attuned voice signal reproduces a dynamic speech directivity of the speaker within the artificial real

Assignees

• Facebook Tech Llc

Inventors

• Robinson Philip

Classifications

• H04S2400/15

  Aspects of sound capture and related signal processing for recording or reproduction · CPC title

• H04S2400/11

  Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) · CPC title

• H04S7/305

  Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence) · CPC title

• H04S7/304 Primary

  For headphones · CPC title

• G10L21/003

  Changing voice quality, e.g. pitch or formants · CPC title