Automated real time interpreter service

Patent metadata

Field	Value
Publication number	US-2018253992-A1
Application number	US-201715730176-A
Country	US
Kind code	A1
Filing date	Oct 11, 2017
Priority date	Mar 3, 2017
Publication date	Sep 6, 2018
Grant date	—

Abstract

Aspects of the present disclosure relate to systems and methods that aid users with hearing and/or speech impediments to have a conversation with a remote phone number without human assistance. In one aspect, an application residing on a user's device, such as a smartphone, tablet computer, laptop, etc., may be used to initiate a phone call to a recipient. Upon initiating the phone call locally, a service residing on the server may receive a request to initiate a connection to the recipient. Once the recipient answers, the user may converse with the recipient by providing text input to their local app. The text input may be transmitted to the service. The service may use a text to speech converter to translate the received text to speech that can be delivered to the recipient.

First claim

What is claimed is:

1 . A method comprising:
receiving, during a call, first data from a device; determining if the first data was received from the interpreter application; when the data was received from an interpreter application, converting the first data from text to speech; and providing the speech to a recipient device.

2 . The method of claim 1 , further comprising:
receiving, during the call, second data; determining the source of the second data; when the source of the second data is the recipient device, converting the second data from speech to text data; and sending the text data to the interpreter application.

3 . The method of claim 2 , wherein the second data represents speech from a speaker associated with the recipient device, and wherein the method further comprises processing the speech to determine a gender of the speaker.

4 . The method of claim 3 , further processing the speech to determine a current emotional state of the speaker.

5 . The method of claim 2 , wherein the second data represents speech from at least two speakers associated with the recipient device.

6 . The method of claim 5 , wherein the method further comprises:
associating a first part of the second data with a first speaker; and associating a second part of the second data with a second speaker.

7 . The method of claim 6 , wherein converting the second data from speech to text data further comprises associating a first speaker identifier identifying the first speaker with first text data and associating a second speaker identifier identifying the second speaker with second text data.

8 . The method of claim 1 , wherein the interpreter application is resident on the device.

9 . A system comprising:
at least one processor; and memory encoding computer executable instructions that, when executed by the at least one processor, perform a method comprising: receiving, at an interpreter service, a request to establish a call from an interpreter application; in response to receiving the call from the interpreter application, initiating the call with a recipient device; upon establishing the call, sending an initial consent message to the recipient device, wherein the initial consent message informs a recipient of the interpreter service; and upon receiving an indication of consent from the recipient device, establishing communication between the interpreter application and the recipient device.

10 . The system of claim 9 , wherein the method further comprises:
upon establishing communication between the interpreter application and the recipient device, receiving first call data; determining if the first call data was received from the interpreter application; when the data was received from the interpreter application, converting the first data from text to speech; and providing the speech to the recipient device.

11 . The system of claim 10 , wherein the method further comprises:
receiving second call data; determining the source of the second call data; when the source of the second call data is the recipient device, converting the second data from speech to text data; and sending the text data to the interpreter application.

12 . The system of claim 11 , wherein the second call data represents speech from a speaker associated with the recipient device, and wherein the method further comprises processing the speech to determine a gender of the speaker.

13 . The system of claim 12 , wherein the method further comprises processing the speech to determine a current emotional state of the speaker.

14 . The system of claim 11 , wherein the second call data represents speech from at least two speakers.

15 . The system of claim 14 , wherein the method further comprises:
associating a first part of the second call data with a first speaker; and associating a second part of the second call data with a second speaker.

16 . The system of claim 15 , wherein converting the second call data from speech to text data further comprises associating a first speaker identifier identifying the first speaker with first text data and associating a second speaker identifier identifying the second speaker with second text data.

17 . A system comprising:
a first device comprising: at least a first processor; and a first memory encoding first computer executable instructions that, when executed by the at least first processor, configure the at least first processor to:
initiate an interpreter application; and
send, to an interpretation service resident on a remote device, a request to establish a call with a recipient device; and
a remote device comprising: at least a second processor; and a first memory encoding first computer executable instructions that, when executed by the at least second processor, configure the at least second processor to:
receive the request from the first device;
in response to receiving the request; initiate the call with the recipient device;
upon establishing the call, send an initial consent message to the recipient device, wherein the initial consent message informs a recipient of the interpreter service; and
upon receiving an indication of consent from the recipient device, establish communication between the interpreter application and the recipient device.

18 . The system of claim 17 , wherein the first device comprises one of:
a smartphone; a tablet; and a computing device.

19 . The system of claim 17 , wherein the recipient device is a telephone.

20 . The system of claim 17 , wherein the remote device receives the request via a network connection.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Koul Anirudh
• Kasam Meher
• Johnston Michael
• Machanavajhala Swetha
• Halper Elizabeth

Classifications

• H04M3/42382

  Text-based messaging services in telephone networks such as PSTN/ISDN, e.g. User-to-User Signalling or Short Message Service for fixed networks · CPC title

• G09B21/009 Primary

  Teaching or communicating with deaf persons (G09B21/04 takes precedence) · CPC title

• H04M3/42391

  where the subscribers are hearing-impaired persons, e.g. telephone devices for the deaf · CPC title

• H04M1/2475

  for a hearing impaired user · CPC title

• G09B5/06

  with both visual and audible presentation of the material to be studied · CPC title