Recording meeting audio via multiple individual smartphones

Patent metadata

Field	Value
Publication number	US-10171908-B1
Application number	US-201615214559-A
Country	US
Kind code	B1
Filing date	Jul 20, 2016
Priority date	Jul 27, 2015
Publication date	Jan 1, 2019
Grant date	Jan 1, 2019

Abstract

Recording audio information from a meeting includes determining which audio input audio device (smartphones) correspond to which meeting participant, measuring volume levels in response to each of the meeting participants speaking, identifying one of the participants is speaking based on stored voice profiles and/or relative volume levels at each of the smartphones, recording on a first channel audio input at a first smartphone corresponding to the speaker, identifying another one of the participants is speaking based on stored voice profiles and/or relative volume levels at each of the smartphones, recording on a second channel, separate from the first channel, audio input at a second smartphone corresponding to the other speaker, and merging the first and second channels to provide a storyboard that includes audio input from the channels and identification of speakers based on which specific ones of the channels contains the audio input.

First claim

What is claimed is:

1. A method of recording audio information from a meeting having a plurality of participants including a first participant and a second participant, comprising:
at a computing system with one or more processors and memory:
establishing a first connection with a first audio input device of a plurality of audio input devices, the first connection configured to enable the computing system to receive a first audio stream recorded by the first audio input device;
establishing a second connection with a second audio input device of the plurality of audio input devices, the second connection configured to enable the computing system to receive a second audio stream recorded by the second audio input device;
determining that the first and second audio input devices correspond respectively to the first and second participants;
receiving the first and second audio streams during the meeting;
measuring relative volume levels of the first and second audio streams;
identifying from the first audio stream first audio fragments corresponding to speech by the first participant based on: (i) a stored voice profile of the first participant, or (ii) the relative volume levels;
storing as a first audio channel the first audio fragments;
identifying from the second audio stream second audio fragments corresponding to speech by the second participant based on: (i) a stored voice profile of the second participant, or (ii) the relative volume levels;
storing as a second audio channel the second audio fragments, the first and second audio channels being separate from each other and being associated with the first and second participants, respectively;
wherein, in response to the first and second participants speaking at the same time, storing the first audio fragments and the second audio fragments includes:
simultaneously storing the first audio fragments as the first audio channel and the second audio fragments as the second audio channel; and
filtering the first audio fragments and the second audio fragments to separate speech by the first participant from speech by the second participant; and

providing, at the computing system, a storyboard audio channel that includes the first and second audio fragments and identifies the first and second participants as speakers corresponding to the first and second audio fragments, respectively, wherein the identifying is based on which of the first and second audio channels contains the first and second audio fragments.

2. A method, according to claim 1 , wherein determining that the first and second audio input devices correspond respectively to the first and second participants is based on which of the plurality of participants owns the first and second audio input devices.

3. A method, according to claim 1 , wherein at least one of the first and second audio input devices is a smartphone.

4. A method, according to claim 1 , further comprising:
establishing the first and second connections prior to the meeting; and
equalizing sound detection levels in the first and second audio input devices.

5. A method, according to claim 1 , wherein filtering the first and second audio fragments is based on a distance related volume weakening coefficient, signal latency between the first and second audio input devices, or ambient noise.

6. A method, according to claim 1 , further comprising:
adding a voice annotation to the storyboard channel.

7. A method, according to claim 6 , wherein the annotation is added following the meeting.

8. A method, according to claim 6 , wherein the annotation is added by one of the plurality of participants.

9. A method, according to claim 6 , wherein the annotation is related to a specific audio fragment, or the annotation is commentary for an entire meeting.

10. A method, according to claim 1 , further comprising:
adding a pre-recorded introduction to the storyboard channel for at least one of the plurality of participants.

11. A method, according to claim 1 , wherein a visual signal is provided on the first or second audio input device.

12. A method, according to claim 11 , wherein the first or second participant responds to the visual signal provided on the first or second audio input device, respectively, to confirm whether the first or second participant is currently speaking.

13. A method, according to claim 1 , further comprising transcribing at least a portion of the storyboard channel using voice-to-text transcription.

14. A method, according to claim 1 , wherein the first connection is established with the first audio input device, the first audio input device being located in a first office, and the second connection is established with the second audio input device, the second audio device being located in a second office remote from the first office.

15. A non-transitory computer-readable medium storing one or more programs configured for execution by a computer system, the one or more programs including instructions for:
establishing a first connection with a first audio input device of a plurality of audio input devices, the first connection configured to enable the computing system to receive a first audio stream recorded by the first audio input device;
establishing a second connection with a second audio input device of the plurality of audio input devices, the second connection configured to enable the computing system to receive a second audio stream recorded by the second audio input device;
determining that the first and second audio input devices correspond respectively to the first and second participants;
receiving the first and second audio streams during the meeting;
measuring relative volume levels of the first and second audio streams;
identifying from the first audio stream first audio fragments corresponding to speech by the first participant based on: (i) a stored voice profile of the first participant, or (ii) the relative volume levels;
storing as a first audio channel the first audio fragments;
identifying from the second audio stream second audio fragments corresponding to speech by the second participant based on: (i) a stored voice profile of the second participant, or (ii) the relative volume levels;
storing as a second audio channel the second audio fragments, the first and second audio channels being separate from each other and being associated with the first and second participants, respectively;
wherein, in response to the first and second participants speaking at the same time, storing the first audio fragments and the second audio fragments includes:
simultaneously storing the first audio fragments as the first audio channel and the second audio fragments as the second audio channel; and
filtering the first audio fragments and the second audio fragments to separate speech by the first participant from speech by the second participant; and

providing, at the computing system, a storyboard audio channel that includes the first and second audio fragments and identifies the first and second participants as speakers corresponding to the first and second audio fragments, respectively, wherein the identifying is based on which of the first and second audio channels contains the first and second audio fragments.

16. A non-transitory computer-readable medium, according to claim 15 , wherein determining that the first and second audio input devices correspond respectively to the first and second participants is based on which of the plurality of participants owns the first and second audio input devices.

17. A non-transitory computer-readable medi

Assignees

• Evernote Corp

Inventors

• Sinkov Andrew
• Pashintsev Alexander

Classifications

• H04R3/005 Primary

  for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407) · CPC title

• G10L2021/02087

  the noise being separate speech, e.g. cocktail party · CPC title

• G10L25/51

  for comparison or discrimination · CPC title

• G10L21/028

  using properties of sound source · CPC title

• G10L17/00

  Speaker identification or verification techniques · CPC title