Session speech-to-text conversion

Patent metadata

Field	Value
Publication number	US-10179291-B2
Application number	US-201715491818-A
Country	US
Kind code	B2
Filing date	Apr 19, 2017
Priority date	Dec 9, 2016
Publication date	Jan 15, 2019
Grant date	Jan 15, 2019

Abstract

Examples described herein provide various devices that enable users to participate in a session. The examples allow a user that is unable to hear, or that is incapable of hearing, to participate in an in-session voice chat by receiving text messages that have been converted from speech to text (e.g., transcribed messages). The user enables a speech-to-text conversion feature on his or her own device. Based on the enabled feature, a notification can be sent to other devices participating in the session. Based on the notification, the other devices are tasked with capturing utterances and sending corresponding speech data to a speech-to-text conversion device so that voice communications can be converted to text messages. Subsequent to conversion, the text messages can be displayed in a chat interface presented as part of the displayed content for the session.

First claim

What is claimed is:

1. A device for displaying text message data that has been converted from speech to text within a session, the device comprising:
one or more processing units; and
a computer-readable medium having encoded thereon computer-executable instructions to configure the one or more processing units to:
receive input that enables a feature that converts speech to text;
send data indicating that the feature has been enabled to other devices participating in the session;
receive, based at least in part on the enabled feature, the text message data that has been converted from speech to text;
cause session content of the session to be displayed in a user interface on a display screen associated with the device, the session content including a chat interface that includes chat content that is rendered by a platform component of the device and game content that is rendered by a title, wherein the chat content is associated with the received text message data and the chat interface is rendered as an overlay layer on top of the game content;
determine, based at least in part on the game content displayed in the user interface, a focus area of the user interface, a size of the focus area being less than a size of the user interface;
select a position for the chat interface within the overlay layer that is outside the focus area; and
cause the chat interface to be displayed in the position selected to avoid obstructing participant interaction with the game content.

2. The device of claim 1 , wherein the focus area of the user interface is further determined based on an eye gaze of a session participant.

3. The device of claim 1 , wherein the position selected is one of a predetermined number of preconfigured positions.

4. The device of claim 1 , wherein the instructions further configure the one or more processing units to:
update, based at least in part on updated game content displayed in the user interface at a later time, the focus area of the user interface;
select another position for the chat interface within the overlay layer that is outside the updated focus area; and
cause the chat interface to be displayed in the other position selected to avoid obstructing participant interaction with the updated game content.

5. The device of claim 1 , wherein the instructions further configure the one or more processing units to:
notify the title of reception of the text message data;
enable the title to select the position for the chat interface within the overlay layer; and
receive, from the title, a selection of the position for the chat interface within the overlay layer.

6. The device of claim 1 , wherein the other devices are prohibited from outputting information related to the enabled feature to session participants of the other devices.

7. The device of claim 1 , wherein the instructions further configure the one or more processing units to visually distinguish, in the chat interface using icons, between individual text messages that have been converted from speech to text and one or more other messages that were originally input as text and that have not been converted from speech to text.

8. The device of claim 1 , wherein the instructions further configure the one or more processing units to:
determine that no communication activity has occurred for a predetermined inactivity time period; and
remove the chat interface from the user interface based at least in part on the determining that no communication activity has occurred for the predetermined inactivity time period.

9. The device of claim 1 , wherein the instructions further configure the one or more processing units to format the chat interface so that participant interaction is prohibited.

10. A method comprising:
receiving, at a device, input that enables a feature that converts speech to text;
sending data indicating that the feature has been enabled to other devices participating in a session;
receiving, based at least in part on the enabled feature, a text message data that has been converted from speech to text;
causing, by one or more processing units, session content of the session to be displayed in a user interface on a display screen associated with the device, the session content including a chat interface that includes chat content that is rendered by a platform component of the device and game content that is rendered by a title, wherein the chat content is associated with the received text message data and the chat interface is rendered as an overlay layer on top of the game content;
determining, based at least in part on the game content displayed in the user interface, a focus area of the user interface, a size of the focus area being less than a size of the user interface;
selecting a position for the chat interface within the overlay layer that is outside the focus area; and
causing the chat interface to be displayed in the position selected to avoid obstructing participant interaction with the game content.

11. The method of claim 10 , wherein the focus area of the user interface is further determined based on an eye gaze of a session participant.

12. The method of claim 10 , wherein the position selected is one of a predetermined number of preconfigured positions.

13. The method of claim 10 , further comprising:
updating, based at least in part on updated game content displayed in the user interface at a later time, the focus area of the user interface;
selecting another position for the chat interface within the overlay layer that is outside the updated focus area; and
causing the chat interface to be displayed in the other position selected to avoid obstructing participant interaction with the updated game content.

14. The method of claim 10 , further comprising:
notifying the title of reception of the text message data;
enabling the title to select the position for the chat interface within the overlay layer; and
receiving, from the title, a selection of the position for the chat interface within the overlay layer.

15. The method of claim 10 , wherein the other devices are prohibited from outputting information related to the enabled feature to session participants of the other devices.

16. The method of claim 10 , further comprising visually distinguishing, in the chat interface using icons, between individual text messages that have been converted from speech to text and one or more other messages that were originally input as text and that have not been converted from speech to text.

17. The method of claim 10 , further comprising:
determining that no communication activity has occurred for a predetermined inactivity time period; and
removing the chat interface from the user interface based at least in part on the determining that no communication activity has occurred for the predetermined inactivity time period.

18. The method of claim 10 , further comprising formatting the chat interface so that participant interaction is prohibited.

19. One or more computer storage media storing instructions that, when executed by one or more processing units, cause a device to perform operations comprising:
receiving input that enables a feature that converts speech to text;
sending data indicating that the feature has been enabled to other devices participating in a session;
receiving, based at least in part on the enabled feature, a text message data that has been converted from speech to text;
causing session content of the session to be displayed in a user interface on a display screen associated with

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Thomas Evelyn R
• Kepner Jason
• Ruzyski David
• Dewey Jeremy L
• Hall Martin
• Mccammon Joshua R
• Hirve Kedar
• Giscombe Stephen E
• St Onge Travis
• Cardwell Aaron
• Labunka Matthew
• Beltrami Luca
• Brodie Heather Grantham

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/0484

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title

• A63F2300/6072

  of an input signal, e.g. pitch and rhythm extraction, voice recognition · CPC title

• A63F13/53

  involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game · CPC title

• A63F2300/308

  Details of the user interface · CPC title