Customized dialogue support

Patent metadata

Field	Value
Publication number	US-2024029725-A1
Application number	US-202217870498-A
Country	US
Kind code	A1
Filing date	Jul 21, 2022
Priority date	Jul 21, 2022
Publication date	Jan 25, 2024
Grant date	—

Abstract

Systems and methods for customized dialogue support in virtual environments are provided. Dialogue maps stored in memory may specify dialogue triggers each associated with a corresponding dialogue instruction. Data regarding an interactive session associated with a user device may be monitored based on one or more of the stored dialogue maps. The presence of one of the dialogue triggers specified by the one or more dialogue maps may be detected based on the monitored data. Customized dialogue output may be generated in response to the detected dialogue trigger and based on the dialogue instruction corresponding to the detected dialogue trigger. The customized dialogue output may be provided to the interactive session in real-time with detection of the detected dialogue trigger.

First claim

1 . A method for customized dialogue support in virtual environments, the method comprising:
storing a plurality of dialogue maps in memory, each dialogue map specifying a plurality of dialogue triggers each associated with a corresponding dialogue instruction; monitoring data regarding an interactive session associated with a user device based on one or more of the stored dialogue maps; applying a language usage model to analyze the data associated with the interactive session and data regarding a user associated with the user device, wherein applying the language usage model results in a prediction of when one of the dialogue triggers specified by the one or more dialogue maps is likely to occur; generating customized dialogue output in response to the predicted dialogue trigger, wherein generating the customized dialogue output is based on the dialogue instruction corresponding to the predicted dialogue trigger; and providing the customized dialogue output to the interactive session in real-time in accordance with when the predicted dialogue trigger is predicted to Occur.

2 . The method of claim 1 , wherein the language usage model is a machine-learning model continually refined to analyze speech patterns for at least one of different users and different game titles.

3 . The method of claim 1 , wherein the language usage model is specific to the user of the user device, and wherein the prediction of when the predicted dialogue trigger is likely to occur is based on comparing a predetermined pattern of speech associated with the user to speech of the user during the interactive session.

4 . The method of claim 1 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify an audio stream of speech by the user of the user device, and wherein the modified audio stream is provided to one or more other user devices in the interactive session in place of the audio stream.

5 . The method of claim 1 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify a display of the user device to include one or more prompts associated with the predicted dialogue trigger.

6 . The method of claim 5 , wherein the prompts are selectable based on gaze data from gaze-tracking of the user within the virtual environment.

7 . The method of claim 1 , wherein at least one of the dialogue maps is specific to a predetermined theme, and wherein the dialogue triggers and the corresponding dialogue instruction include words and phrases associated with the predetermined theme.

8 . The method of claim 1 , wherein at least one of the dialogue maps specifies a dialogue trigger that includes at least one of gesture, controller input, or custom shortcut input.

9 . The method of claim 1 , wherein at least one of the dialogue maps specifies a dialogue instruction executable to play a pre-recorded audio clip in the interactive session.

10 . The method of claim 1 , further comprising customizing one of the dialogue maps for a user of the user device, wherein customizing the dialogue map includes:
calibrating for one or more identified speech patterns and abilities of the user; and defining one or more of the dialogue triggers based on the calibration.

11 . The method of claim 1 , further comprising preconfiguring the corresponding dialogue instruction in accordance with a custom shortcut command, and defining one or more of the dialogue triggers based on the custom shortcut command.

12 . A system for customized dialogue support in virtual environments, the system comprising:
memory that stores a plurality of dialogue maps, each dialogue map specifying a plurality of dialogue triggers each associated with a corresponding dialogue instruction; a communication interface that communicates over a communication network, wherein the communication interface receives data regarding an interactive session associated with a user device; and a processor that executes instructions stored in memory, wherein the processor executes the instructions to:
monitor the received data regarding the interactive session based on one or more of the stored dialogue maps;
apply a language usage model to analyze the data associated with the interactive session and data regarding a user associated with the user device, wherein applying the language usage model results in a prediction of when one of the dialogue triggers specified by the one or more dialogue maps is likely to occur; and
generate customized dialogue output in response to the predicted dialogue trigger, wherein generating the customized dialogue output is based on the dialogue instruction corresponding to the predicted dialogue trigger;
wherein the communication interface provides the customized dialogue output to the interactive session in real-time in accordance with when the predicted dialogue trigger is predicted to occur.

13 . The system of claim 12 , wherein the language usage model is a machine-learning model continually refined to analyze speech patterns for at least one of different users and different game titles.

14 . The system of claim 12 , wherein the language usage model is specific to the user of the user device, and wherein the prediction of when the predicted dialogue trigger is likely to occur is based on comparing a predetermined pattern of speech associated with the user to speech of the user during the interactive session.

15 . The system of claim 12 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify an audio stream of speech by the user of the user device, and wherein the modified audio stream is provided to one or more other user devices in the interactive session in place of the audio stream.

16 . The system of claim 12 , wherein the dialogue instruction corresponding to the predicted dialogue trigger is executable to modify a display of the user device to include one or more prompts associated with the predicted dialogue trigger.

17 . The system of claim 16 , wherein the prompts are selectable based on gaze data from gaze-tracking of the user within the virtual environment.

18 . The system of claim 12 , wherein at least one of the dialogue maps is specific to a predetermined theme, and wherein the dialogue triggers and the corresponding dialogue instruction include words and phrases associated with the predetermined theme.

19 . The system of claim 12 , wherein at least one of the dialogue maps specifies a dialogue trigger that includes at least one of gesture, controller input, or custom shortcut input.

20 . The system of claim 12 , wherein at least one of the dialogue maps specifies a dialogue instruction executable to play a pre-recorded audio clip in the interactive session.

21 . The system of claim 12 , wherein the processor executes further instructions to customize one of the dialogue maps for a user of the user device, wherein the processor customizes the dialogue map by:
calibrating for one or more identified speech patterns and abilities of the user; and defining one or more of the dialogue triggers based on the calibration.

22 . The system of claim 12 , wherein the processor executes further instructions to preconfigure the corresponding dialogue instruction in accordance with a custom shortcut command, and define one or more of the dialogue triggers based on the custom shortcut command.

23 . A non-transitory, computer-readable storage med

Assignees

• Sony Interactive Entertainment LLC
• Sony Interactive Entertainment Inc

Inventors

• Fryer-Mcculloch Morgan
• Dorn Victoria
• Powell Brielle
• Osman Steven
• Norton Geoff

Classifications

• G10L15/22 Primary

  Procedures used during a speech recognition process, e.g. man-machine dialogue · CPC title

• G10L15/18

  using natural language modelling · CPC title

• G10L15/30

  Distributed recognition, e.g. in client-server systems, for mobile phones or network applications · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G10L25/51 Primary

  for comparison or discrimination · CPC title