Automated real time interpreter service

Patent metadata

Field	Value
Publication number	US-11699360-B2
Application number	US-202017106901-A
Country	US
Kind code	B2
Filing date	Nov 30, 2020
Priority date	Mar 3, 2017
Publication date	Jul 11, 2023
Grant date	Jul 11, 2023

Abstract

Aspects of the present disclosure relate to systems and methods that aid users with hearing and/or speech impediments to have a conversation with a remote phone number without human assistance. In one aspect, an application residing on a user's device, such as a smartphone, tablet computer, laptop, etc., may be used to initiate a phone call to a recipient. Upon initiating the phone call locally, a service residing on the server may receive a request to initiate a connection to the recipient. Once the recipient answers, the user may converse with the recipient by providing text input to their local app. The text input may be transmitted to the service. The service may use a text to speech converter to translate the received text to speech that can be delivered to the recipient.

First claim

What is claimed is:

1. A method of a real-time communication initiated by a voice caller to a non-voice callee, the method comprising:
receiving, by an interpreter application used by the non-voice callee, voice call initiation data from the voice caller, wherein the interpreter application communicates the voice call initiation data to the non-voice callee based on text data;
establishing a voice phone call with the voice caller based on the voice call initiation data;
receiving, by the interpreter application, speech data from the voice caller;
analyzing the speech data to determine a feature of the speech data, wherein the feature comprises one or more of: a gender of the voice caller, an identity of the voice caller, an emotional state of the voice caller, or an amplitude;
converting, by the interpreter application, the speech data to text data; and
providing the text data to the non-voice callee with one or more indications based on the determined feature.

2. The method of claim 1 , wherein the interpreter application is associated with a service.

3. The method of claim 1 , further comprising:
analyzing the speech data to determine a gender of the voice caller; and
providing the text data to the non-voice callee with an indication of the gender of the voice caller.

4. The method of claim 3 , wherein the indication of the gender is a font of the text data.

5. The method of claim 1 , further comprising:
analyzing the speech data to determine an identity of the voice caller; and
providing the text data to the non-voice callee with an indication of the identity of the voice caller.

6. The method of claim 1 , further comprising:
analyzing the speech data to determine an emotional state of the voice caller; and
modifying the text data to indicate the emotional state of the voice caller.

7. The method of claim 1 , further comprising:
analyzing the speech data to determine an amplitude; and
modifying the text data to indicate the amplitude of the speech data.

8. A system, comprising:
at least one processor; and
at least one memory storing computer-executable instructions that when executed cause the at least one processor to:
receive, by an interpreter application used by a non-voice callee, voice call initiation data, wherein the voice call initiation data is operable to establish a voice phone call between the non-voice callee and two or more voice callers;
receive, by the interpreter application, speech data from the two or more voice callers;
associate a first portion of the speech data with a first voice caller of the two or more voice callers and a second portion of the speech data with a second voice caller of the two or more voice callers;
receive, by the interpreter application, a first gender of the first voice caller and a second gender of the second voice caller, the first and second genders being based on the speech data;
convert, by the interpreter application, the first portion of the speech data to first text data associated with the first voice caller of the two or more voice callers and the second portion of the speech data to second text data associated with the second voice caller of the two or more voice callers; and
provide the first text data and the second text data to the non-voice callee with a first indication of the first gender of the first voice caller and a second indication of the second gender of the second voice caller.

9. The system of claim 8 , the computer-executable instructions when executed further causing the at least one processor to:
analyze the first portion of the speech data to determine a first identity of the first voice caller and the second portion of the speech data to determine a second identity of the second voice caller; and
provide the first text data with a first voice caller identifier of the first voice caller and the second text data with a second voice caller identifier of the second voice caller.

10. The system of claim 8 , wherein the interpreter application is associated with a service.

11. The system of claim 8 , wherein the voice phone call is a conference call.

12. The system of claim 8 , the computer-executable instructions when executed further causing the at least one processor to:
analyze the first portion of the speech data to determine the first gender of the first voice caller and the second portion of the speech data to determine the second gender of the second voice caller.

13. The system of claim 12 , wherein the first gender is different than the second gender, and wherein the first indication of the first gender is different than the second indication of the second gender.

14. The system of claim 13 , wherein the first indication of the first gender is a first font of the first text data and the second indication of the second gender is a second font of the second text data.

15. The system of claim 12 , wherein the first gender is the same as the second gender, and wherein the first indication of the first gender is the same as the second indication of the second gender.

16. The system of claim 8 , the computer-executable instructions when executed further causing the at least one processor to:
analyze the first portion of the speech data to determine a first emotional state of the first voice caller and the second portion of the speech data to determine a second emotional state of the second voice caller; and
modify the first text data to indicate the first emotional state of the first voice caller and the second text data to indicate the second emotional state of the second voice caller.

17. The system of claim 16 , wherein the first emotional state is different than the second emotional state, and wherein the indication of the first emotional state is different than the indication of the second emotional state.

18. A computer storage medium storing computer-executable instructions that when executed cause at least one processor to:
receive, by an interpreter application used by a non-voice callee, voice call initiation data, wherein the voice call initiation data is operable to establish a voice phone call between the non-voice callee and a voice caller;
receive, by the interpreter application, speech data from the voice caller;
analyze the speech data to determine an identity and a gender of the voice caller;
convert, by the interpreter application, the speech data to text data; and
provide the text data to the non-voice callee with an identifier of the voice caller and in a font associated with the gender of the voice caller.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Koul Anirudh
• Kasam Meher
• Johnston Michael
• Machanavajhala Swetha
• Halper Elizabeth

Classifications

• G09B21/009 Primary

  Teaching or communicating with deaf persons (G09B21/04 takes precedence) · CPC title

• H04M11/02

  with bell or annunciator systems · CPC title

• H04M11/066

  Telephone sets adapted for data transmision · CPC title

• H04M3/42382

  Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks · CPC title

• H04M2201/39

  using speech synthesis · CPC title