Personalized VR controls and communications

Patent metadata

Field	Value
Publication number	US-11992756-B2
Application number	US-202117510586-A
Country	US
Kind code	B2
Filing date	Oct 26, 2021
Priority date	Oct 26, 2021
Publication date	May 28, 2024
Grant date	May 28, 2024

Abstract

Systems and methods for personalized controls and communications in virtual environments are provided. A virtual reality (VR) profile may be stored in memory for a user. Such VR profile may specify a cue associated with custom instructions executable to modify one or more virtual display elements. An interactive session associated with a virtual environment in which the user is participating via a user device may be monitored based on the VR profile stored for the user. The cue specified by the VR profile may be detected as being present in the monitored communication session. The virtual elements may be modified within a presentation of the virtual environment provided to the user device in accordance with the executable instructions associated with the cue specified by the VR profile of the user.

First claim

What is claimed is:

1. A method for personalized controls and communications in virtual environments, the method comprising:
storing a virtual reality (VR) profile in memory for a user, wherein the VR profile specifies a cue associated with custom instructions executable to modify one or more virtual elements;
monitoring an interactive session associated with a current virtual environment in which the user is participating via a user device, wherein monitoring the interactive session is based on the VR profile stored for the user;
detecting that the cue specified by the VR profile is present in the monitored communication session, wherein the specified cue includes text from one or more other devices in the interactive session;
generating voiceover audio based on the text in real-time;
modifying the virtual elements within a presentation of the current virtual environment provided to the user device in accordance with the executable instructions associated with the cue specified by the VR profile of the user, wherein the presentation is modified to include the voiceover audio generated in real-time based on content associated with the one or more other devices; and
providing the modified presentation of the current virtual environment with generated voiceover audio to the user device.

2. The method of claim 1 , wherein the specified cue includes a real-world movement by the user, and wherein detecting the specified cue is based on sensor information captured by the user device.

3. The method of claim 2 , wherein the real-world movement includes a gesture in physical space, and wherein the sensor information is captured by a camera associated with the user device.

4. The method of claim 2 , wherein the real-world movement includes a drawing gesture upon a touchpad of the user device, and wherein the sensor information is captured by the touchpad.

5. The method of claim 1 , wherein at least one of the cue, a type of modification, and the virtual display elements is customizable by the user.

6. The method of claim 1 , further comprising identifying the virtual elements within the current virtual environment based on eye-tracking data indicating that a line-of-sight of the user is directed at the virtual display elements within the virtual environment when the specified cue was detected.

7. The method of claim 6 , wherein the current virtual environment includes one or more user input interfaces, and wherein modifying the virtual elements includes generating typed output when the specified cue is associated with the user input interfaces.

8. The method of claim 1 , wherein the specified cue includes text from one or more other devices in the interactive session, and wherein modifying the virtual elements includes removing the text from the current virtual environment.

9. A system for personalized controls and communications in virtual environments, the system comprising:
memory that stores a virtual reality (VR) profile for a user, wherein the VR profile specifies a cue associated with custom instructions executable to modify one or more virtual elements;
a communication interface that communicates over a communication network, wherein the communication interface receives data regarding an interactive session associated with a current virtual environment in which the user is participating via a user device; and
a processor that executes instructions stored in memory, wherein the processor executes the instructions to:
monitor the interactive session based on the VR profile stored for the user,
detect that the cue specified by the VR profile is present in the monitored communication session, wherein the specified cue includes text from one or more other devices in the interactive session,
generate voiceover audio based on the text in real-time,
modify the virtual elements within a presentation of the current virtual environment provided to the user device in accordance with the executable instructions associated with the cue specified by the VR profile of the user, wherein the presentation is modified to include the voiceover audio generated in real-time based on content associated with the one or more other devices, and
provide the modified presentation of the current virtual environment with generated voiceover audio to the user device.

10. The system of claim 9 , wherein the specified cue includes a real-world movement by the user, and wherein the processor detects the specified cue based on sensor information captured by the user device.

11. The system of claim 10 , wherein the real-world movement includes a gesture in physical space, and wherein the sensor information is captured by a camera associated with the user device.

12. The system of claim 10 , wherein the real-world movement includes a drawing gesture upon a touchpad of the user device, and wherein the sensor information is captured by the touchpad.

13. The system of claim 9 , wherein at least one of the cue, a type of modification, and the virtual elements is customizable by the user.

14. The system of claim 9 , wherein the processor executes further instructions to identify the virtual elements within the current virtual environment based on eye-tracking data indicating that a line-of-sight of the user is directed at the virtual elements within the virtual environment when the specified cue was detected.

15. The system of claim 14 , wherein the current virtual environment includes one or more user input interfaces, and wherein the processor modifies the virtual elements by generating typed output when the specified cue is associated with the user input interfaces.

16. The system of claim 9 , wherein the specified cue includes text from one or more other devices in the interactive session, and wherein the processor modifies the virtual elements by removing the text from the current virtual environment.

17. A non-transitory, computer-readable storage medium, having embodied thereon a program executable by a processor to perform a method for personalized controls and communications in virtual environments, the method comprising:
storing a virtual reality (VR) profile in memory for a user, wherein the VR profile specifies a cue associated with custom instructions executable to modify one or more virtual elements;
monitoring an interactive session associated with a current virtual environment in which the user is participating via a user device, wherein monitoring the interactive session is based on the VR profile stored for the user;
detecting that the cue specified by the VR profile is present in the monitored communication session, wherein the specified cue includes text from one or more other devices in the interactive session;
generating voiceover audio based on the text in real-time;
modifying the virtual elements within a presentation of the current virtual environment provided to the user device in accordance with the executable instructions associated with the cue specified by the VR profile of the user, wherein the presentation is modified to include the voiceover audio generated in real-time based on content associated with the one or more other devices; and
providing the modified presentation of the current virtual environment with generated voiceover audio to the user device.

Assignees

• Sony Interactive Entertainment LLC

Inventors

• Young Kristine
• Manzon-Gutzman Tatianna
• Yang Rui
• Chien Hsin-Yi

Classifications

• A63F13/42 Primary

  by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle · CPC title

• A63F13/213 Primary

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title