Graphical style modification for video games using machine learning

Patent metadata

Field	Value
Publication number	US-10977872-B2
Application number	US-201816177241-A
Country	US
Kind code	B2
Filing date	Oct 31, 2018
Priority date	Oct 31, 2018
Publication date	Apr 13, 2021
Grant date	Apr 13, 2021

Abstract

Graphical style modification may be implemented using machine learning. A color accommodation module receives an image frame from a host system and generates a color-adapted version of the image frame. A Graphical Style Modification module receives a first image frame from a host system and applies a style adapted from a second image frame to the first image frame to generate a style adapted first image frame.

First claim

What is claimed is:

1. A system for enhancing Audio Visual content, the system comprising:
a processor;
a memory coupled to the processor;
non-transitory instructions embedded in memory for a method of Graphical Style Modification comprising
receiving a source image frame from a host system;
applying a style adapted from a target image frame to the source image frame to generate a style adapted source image frame, wherein the source image frame is part of a buffered video stream; and
generating a style adapted video stream wherein generating the style adapted video steam includes applying the style adapted from the target image frame to each image frame in the buffered video stream for a duration of the buffered video stream that is less than or equal to the time it takes for a Graphical Style Modification neural network to apply the style adapted from the target image frame to the source image frame.

2. The system of claim 1 wherein the method for Graphical Style Modification further comprises providing the style adapted source image frame to the host system for display on a display screen.

3. The system of claim 1 wherein the method for Graphical Style Modification module further comprises applying the style adapted from the target image frame only to a portion of the source image frame that is less than the entirety of the source image frame.

4. The system of claim 1 further comprising an accessibility controller coupled to the processor, wherein the accessibility controller is configured to synchronize the output of the processor with one or more other neural network modules and wherein the method for Graphical Style Modification further comprises training a generative-neural network to apply the color style of the target frame to the source frame.

5. The system of claim 4 wherein the one or more other neural network modules includes an action description module configured to describe action occurring within a video that includes the source image frame.

6. The system of claim 1 where in the source and target image frames are video game images.

7. A method for enhancing Audio Visual content, comprising:
receiving a source image frame from a host system;
applying a style adapted from a target image frame to the source image frame to generate a style adapted source image frame with a Graphical Style Modification module wherein the source image frame is part of a buffered video stream; and generating a style adapted video stream wherein generating the style adapted video steam includes applying the style adapted from the target image frame to each image frame in the buffered video stream for a duration of the buffered video stream that is less than or equal to the time it takes for a Graphical Style Modification neural network to apply the style adapted from the target image frame to the source image frame.

8. The method of claim 7 further comprising providing the style adapted source image frame to the host system for display on a display screen with the Graphical Style Modification module.

9. The method of claim 7 wherein applying the style adapted from the target image frame includes applying the style adapted from the target image frame only to a portion of the source image frame that is less than the entirety of the source image frame.

10. The method of claim 7 further comprising synchronizing with an accessibility controller, the output of the Graphical Style Modification module with one or more other neural network modules and wherein the Graphical Style Modification module comprises a generative neural network that is trained to apply the color style of the target image frame to the source image frame.

11. The method of claim 10 wherein the one or more other neural network modules includes an action description module configured to describe action occurring within a video that includes the source image frame.

12. The method of claim 7 wherein the source and target image frames are video game images.

13. A non-transitory computer-readable medium having computer readable instructions embodied therein, the instructions being configured upon execution to implement a method for enhancing Audio Visual content, the method, comprising:
receiving a source image frame from a host system;
applying a style adapted from a target image frame to the source image frame to generate a style adapted source image frame with a Graphical Style Modification module wherein the source image frame is part of a buffered video stream; and
generating a style adapted video stream wherein generating the style adapted video steam includes applying the style adapted from the target image frame to each image frame in the buffered video stream for a duration of the buffered video stream that is less than or equal to the time it takes for a Graphical Style Modification neural network to apply the style adapted from the target image frame to the source image frame.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Krishnamurthy Sudha
• Singh Ashish
• Kumar Naveen
• Adams Justice
• Jati Arindam
• Omote Masanori

Classifications

• G06N3/047

  Probabilistic or stochastic networks · CPC title

• G06N3/044

  Recurrent networks, e.g. Hopfield networks · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06N3/09

  Supervised learning · CPC title