Customized dialogue support

Patent metadata

Field	Value
Publication number	US-12400649-B2
Application number	US-202217870498-A
Country	US
Kind code	B2
Filing date	Jul 21, 2022
Priority date	Jul 21, 2022
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

Systems and methods for customized dialogue support in virtual environments are provided. Dialogue maps stored in memory may specify dialogue triggers each associated with a corresponding dialogue instruction. Data regarding an interactive session associated with a user device may be monitored based on one or more of the stored dialogue maps. The presence of one of the dialogue triggers specified by the one or more dialogue maps may be detected based on the monitored data. Customized dialogue output may be generated in response to the detected dialogue trigger and based on the dialogue instruction corresponding to the detected dialogue trigger. The customized dialogue output may be provided to the interactive session in real-time with detection of the detected dialogue trigger.

First claim

What is claimed is:

1. A method for customized dialogue support in virtual environments, the method comprising:
storing a plurality of dialogue maps in memory, each dialogue map specifying a plurality of dialogue triggers each associated with a corresponding dialogue instruction, and each dialogue map being specific to one or more particular game titles of different video games;
monitoring audio data captured from a user device during a stream of an interactive session of a user playing a video game, wherein the audio data includes dialogue spoken by the user;
selecting one or more dialogue maps based on the dialogue and based on a title of the video game played by the user;
predicting, using a language usage model specific to speech patterns of the user of the user device, that one of the dialogue triggers of the associated dialogue maps is likely to occur based on:
matching one or more speech patterns specified by the language usage model to the dialogue from the user device; and
identifying an upcoming in-game event in the video game where the dialogue trigger is likely to occur;

generating customized dialogue output in response to the predicted dialogue trigger, wherein generating the customized dialogue output is performed in real-time and is based on the dialogue instruction corresponding to the predicted dialogue trigger; and
providing the customized dialogue output at a portion of the interactive session stream upon occurrence of the upcoming in-game event.

2. The method of claim 1 , wherein the language usage model is a machine-learning model continually refined to analyze speech patterns for at least one of different users and different game titles.

3. The method of claim 1 , wherein the prediction of when the predicted dialogue trigger is likely to occur is based on comparing a predetermined pattern of speech associated with the user to speech of the user during the interactive session.

4. The method of claim 1 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify an audio stream of speech by the user of the user device by replacing at least a portion of the audio stream with a replacement portion, and wherein the replacement portion is provided to one or more other user devices in the interactive session in place of the replaced portion of the audio stream upon occurrence of the upcoming in-game event.

5. The method of claim 1 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify a display of the user device to include one or more prompts associated with the predicted dialogue trigger.

6. The method of claim 5 , wherein the prompts are selectable based on gaze data from gaze-tracking of the user within the virtual environment.

7. The method of claim 1 , wherein at least one of the dialogue maps is specific to a predetermined theme, and wherein the dialogue triggers and the corresponding dialogue instruction include words and phrases associated with the predetermined theme.

8. The method of claim 1 , wherein at least one of the dialogue maps specifies a dialogue trigger that includes at least one of gesture, controller input, or custom shortcut input.

9. The method of claim 1 , wherein at least one of the dialogue maps specifies a dialogue instruction executable to play a pre-recorded audio clip in the interactive session.

10. The method of claim 1 , further comprising customizing one of the dialogue maps for the user of the user device, wherein customizing the dialogue map includes:
calibrating for one or more identified speech patterns and abilities of the user; and
defining one or more of the dialogue triggers based on the calibration.

11. The method of claim 1 , further comprising preconfiguring the corresponding dialogue instruction in accordance with a custom shortcut command, and defining one or more of the dialogue triggers based on the custom shortcut command.

12. A system for customized dialogue support in virtual environments, the system comprising:
memory that stores a plurality of dialogue maps, each dialogue map specifying a plurality of dialogue triggers each associated with a corresponding dialogue instruction, and each dialogue map being specific to one or more particular game titles of different video games;
a communication interface that communicates over a communication network, wherein the communication interface receives audio data captured from a user device during a stream of an interactive session of a user playing a video game, wherein the audio data includes dialogue spoken by the user; and
a processor that executes instructions stored in memory, wherein the processor executes the instructions to:
monitor the received audio data captured during the stream of the interactive session;
select one or more dialogue maps based on the dialogue and based on a title of the video game played by the user;
predict, using a language usage model specific to speech patterns of the user of the user device, that one of the dialogue triggers of the associated dialogue maps is likely to occur based on:
matching one or more of the speech patterns specified by the language usage model to the dialogue from the user device; and
identifying an upcoming in-game event in the video game where the dialogue trigger is likely to occur; and

generate customized dialogue output in response to the predicted dialogue trigger, wherein generating the customized dialogue output is performed in real-time and is based on the dialogue instruction corresponding to the predicted dialogue trigger;
wherein the communication interface provides the customized dialogue output at a portion of the interactive session stream upon occurrence of the upcoming in-game event.

13. The system of claim 12 , wherein the language usage model is a machine-learning model continually refined to analyze speech patterns for at least one of different users and different game titles.

14. The system of claim 12 , wherein the prediction of when the predicted dialogue trigger is likely to occur is based on comparing a predetermined pattern of speech associated with the user to speech of the user during the interactive session.

15. The system of claim 12 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify an audio stream of speech by the user of the user device by replacing at least a portion of the audio stream with a replacement portion, and wherein the replacement portion is provided to one or more other user devices in the interactive session in place of the replaced portion of the audio stream upon occurrence of the upcoming in-game event.

16. The system of claim 12 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify a display of the user device to include one or more prompts associated with the predicted dialogue trigger.

17. The system of claim 16 , wherein the prompts are selectable based on gaze data from gaze-tracking of the user within the virtual environment.

18. The system of claim 12 , wherein at least one of the dialogue maps is specific to a predetermined theme, and wherein the dialogue triggers and the corresponding dialogue instruction include words and phrases associated with the predetermined theme.

19. The system of claim 12 , wherein at least one of the dialogue maps specifies a dialogue trigger that includes at least one of gesture, controller input, or custom shortcut input.

20. The system of clai

Assignees

• Sony Interactive Entertainment LLC
• Sony Interactive Entertainment Inc

Inventors

• Fryer-Mcculloch Morgan
• Dorn Victoria
• Powell Brielle
• Osman Steven
• Norton Geoff

Classifications

• G10L15/18

  using natural language modelling · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• A63F13/352

  involving special game server arrangements, e.g. regional servers connected to a national server or a plurality of servers managing partitions of the game world · CPC title

• A63F13/424

  involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition · CPC title