Generation of automated message responses

Patent metadata

Field	Value
Publication number	US-11496582-B2
Application number	US-201916455604-A
Country	US
Kind code	B2
Filing date	Jun 27, 2019
Priority date	Sep 26, 2016
Publication date	Nov 8, 2022
Grant date	Nov 8, 2022

Abstract

Systems, methods, and devices for computer-generating responses and sending responses to communications when the recipient of the communication is unavailable are disclosed. An individual may send a message (either audio or text) to a recipient. The recipient may be unavailable to contemporaneously respond to the message (e.g., the recipient may be performing an action that makes is difficult or impractical for the recipient to contemporaneously respond to the audio message). When the recipient is unavailable, a response to the message is generated and sent without receiving an instruction from the recipient to do so. The response may be sent to the message originating individual, and content of the response may thereafter be sent to the recipient to receive feedback regarding the correctness of the response. Alternatively, the response content may first be sent to the recipient to receive the feedback, and thereafter the response may be sent to the message originating individual.

First claim

What is claimed is:

1. A method performed by a computing system, comprising:
receiving first audio data corresponding to a first utterance by a first user operating a first device;
determining that the first utterance represents a message directed to a second user;
determining, based at least in part on the first audio data and contextual information corresponding to a status of the second user, first text responsive to the first utterance;
determining that a first audio response to the first utterance is to be generated based on profile information associated with the second user;
determining that the profile information indicates that speech quality characteristic data is to be used to generate the first audio response;
using the speech quality characteristic data to perform text-to-speech processing on the first text to generate second audio data; and
causing the first device to output audio corresponding to the second audio data.

2. The method of claim 1 , wherein the speech quality characteristic data corresponds to at least one prosodic characteristic for at least one phonetic unit.

3. The method of claim 1 , wherein the speech quality characteristic data corresponds to at least one of a specific gender, a specific accent, a specific speed of speaking, or a distinctive emotive quality.

4. The method of claim 1 , further comprising:
receiving, from a second device, second audio data corresponding to a second utterance by the second user;
determining, based at least in part on the second audio data, second text responsive to the second utterance; and
sending, to the second device, second audio data corresponding to the second text.

5. The method of claim 1 , further comprising:
determining that the first audio data has at least one first speech quality characteristic; and
based at least in part on the first audio data having the at least one first speech quality characteristic, further controlling the text-to-speech processing to cause the first audio data to further have the at least one first speech quality characteristic.

6. The method of claim 1 , further comprising:
determining voice corpus data corresponding to the speech quality characteristic data,
wherein performing the text-to-speech processing uses the voice corpus data.

7. The method of claim 1 , further comprising:
determining parametric feature data corresponding to the speech quality characteristic data,
wherein performing the text-to-speech processing comprises performing speech synthesis using the parametric feature data.

8. A computing system, comprising:
at least one processor; and
at least one computer-readable medium comprising instructions which, when executed by the at least one processor, cause the computing system to:
receive first audio data corresponding to a first utterance by a first user operating a first device,
determine that the first utterance represents a message directed to a second user,
determine, based at least in part on the first audio data and contextual information corresponding to a status of the second user, first text responsive to the first utterance,
determine that a first audio response to the first utterance is to be generated based on profile information associated with the second user,
determine that the profile information indicates that speech quality characteristic data is to be used to generate the first audio response,
use the speech quality characteristic data to perform text-to-speech processing on the first text to generate second audio data, and
cause the first device to output audio corresponding to the second audio data.

9. The computing system of claim 8 , wherein the speech quality characteristic data corresponds to at least one prosodic characteristic for at least one phonetic unit.

10. The computing system of claim 8 , wherein the speech quality characteristic data corresponds to at least one of a specific gender, a specific accent, a specific speed of speaking, or a distinctive emotive quality.

11. The computing system of claim 8 , wherein the at least one computer-readable medium comprises further instructions which, when executed by the at least one processor, further cause the computing system to:
receive, from a second device, second audio data corresponding to a second utterance by the second user;
determine, based at least in part on the second audio data, second text responsive to the second utterance; and
send, to the second device, second audio data corresponding to the second text.

12. The computing system of claim 8 , wherein the at least one computer-readable medium comprises further instructions which, when executed by the at least one processor, further cause the computing system to:
determine that the first audio data has at least one first speech quality characteristic; and
based at least in part on the first audio data having the at least one first speech quality characteristic, further control the text-to-speech processing to cause the first audio data to further have the at least one first speech quality characteristic.

13. The computing system of claim 8 , wherein the at least one computer-readable medium comprises further instructions which, when executed by the at least one processor, further cause the computing system to:
determine voice corpus data corresponding to the speech quality characteristic data, wherein the voice corpus data is used to perform the text-to-speech processing.

14. The computing system of claim 8 , wherein the at least one computer-readable medium comprises further instructions which, when executed by the at least one processor, further cause the computing system to:
determine parametric feature data corresponding to the speech quality characteristic data,
wherein the text-to-speech processing is performed at least in part by performing speech synthesis using the parametric feature data.

Assignees

• Amazon Tech Inc

Inventors

• Rastrow Ariya
• Hardie Tony
• Prasad Rohit

Classifications

• H04L51/02

  using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages · CPC title

• H04M3/527

  Centralised call answering arrangements not requiring operator intervention · CPC title

• H04L67/306 Primary

  User profiles · CPC title

• H04M2201/39

  using speech synthesis · CPC title

• H04L51/043

  using or handling presence information · CPC title