Beginning scan-out process at flip-time for cloud gaming applications

Patent metadata

Field	Value
Publication number	US-12370442-B2
Application number	US-202217948985-A
Country	US
Kind code	B2
Filing date	Sep 20, 2022
Priority date	Oct 1, 2019
Publication date	Jul 29, 2025
Grant date	Jul 29, 2025

Abstract

A method for cloud gaming. The method including generating a video frame when executing a video game at a server. The method including performing a scan-out process by scanning the video frame scanline-by-scanline and one or more user interface features scanline-by-scanline to one or more input frame buffers, and compositing and blending the video frame and the one or more user interface features into a modified video frame. The method including scanning in the scan-out process the modified video frame to an encoder at the server scanline-by-scanline. The method including beginning in the scan-out process scanning the video frame and the one or more user interface features to the one or more input frame buffers at a corresponding flip-time for the video frame.

First claim

What is claimed is:

1. A method, comprising:
generating a video frame within a frame period when executing a video game at a server, wherein generating the video frame comprises executing, at a flip-time that occurs before a next occurrence of a server vertical synchronization (VSYNC) signal, a command that indicates that the video frame has fully rendered; and
performing a scan-out process for the video frame to deliver the video frame to an encoder that is configured to compress the video frame, wherein the performing the scan-out process for the video frame begins at the flip-time.

2. The method of claim 1 , further comprising:
scanning the video frame at a scan-out rate that is higher than a frame rate used for generating a plurality of video frames when executing the video game; and
beginning to encode the video frame before the next occurrence of the server VSYNC signal.

3. The method of claim 1 , wherein the performing the scan-out process for the video frame includes:
scanning the video frame scanline-by-scanline to an input frame buffer.

4. The method of claim 3 , further comprising:
beginning to scan the video frame to the encoder before fully receiving the video frame at the input frame buffer.

5. The method of claim 3 , further comprising:
performing at least one of the following on the video frame received at the input frame buffer to generate a modified video frame:
decompressing DCC compressed surface, or
resolution scaling, or
color space conversion, or
degamma, or
HDR expansion, or
gamut remap, or
LUT shaping, or
tone mapping, or
blending gamma; and

scanning the modified video frame to the encoder.

6. The method of claim 1 ,
wherein the flip-time is a command in a command buffer that when executed by a graphics processing unit (GPU) indicates that the GPU has fully rendered the video frame.

7. The method of claim 1 , further comprising:
beginning to encode the video frame before completing the scan-out process for the video frame.

8. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a system to perform operations comprising:
generating a video frame within a frame period when executing a video game at a server, wherein generating the video frame comprises executing, at a flip-time that occurs before a next occurrence of a server vertical synchronization (VSYNC) signal, a command that indicates that the video frame has fully rendered; and
performing a scan-out process for the video frame to deliver the video frame to an encoder that is configured to compress the video frame, wherein the performing the scan-out process for the video frame begins at the flip-time.

9. The non-transitory computer-readable medium of claim 8 , the operations further comprising:
scanning the video frame at a scan-out rate that is higher than a frame rate used for generating a plurality of video frames when executing the video game; and
beginning to encode the video frame before the next occurrence of the server VSYNC signal.

10. The non-transitory computer-readable medium of claim 8 , wherein the performing the scan-out process for the video frame comprises:
scanning the video frame scanline-by-scanline to an input frame buffer.

11. The non-transitory computer-readable medium of claim 10 , the operations further comprising:
beginning to scan the video frame to the encoder before fully receiving the video frame at the input frame buffer.

12. The non-transitory computer-readable medium of claim 10 , the operations further comprising:
performing at least one of the following on the video frame received at the input frame buffer to generate a modified video frame:
decompressing DCC compressed surface, or
resolution scaling, or
color space conversion, or
degamma, or
HDR expansion, or
gamut remap, or
LUT shaping, or
tone mapping, or
blending gamma; and

scanning the modified video frame to the encoder.

13. The non-transitory computer-readable medium of claim 8 ,
wherein in the method the flip-time is a command in a command buffer that when executed by a graphics processing unit (GPU) indicates that the GPU has fully rendered the video frame.

14. The non-transitory computer-readable medium of claim 8 , the operations further comprising:
beginning to encode the video frame before completing the scan-out process for the video frame.

15. A computer system comprising:
a processor; and
memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method comprising:
generating a video frame within a frame period when executing a video game at a server, wherein generating the video frame comprises executing, at a flip-time that occurs before a next occurrence of a server vertical synchronization (VSYNC) signal, a command that indicates that the video frame has fully rendered;
performing a scan-out process for the video frame to deliver the video frame to an encoder that is configured to compress the video frame, wherein the performing
the scan-out process for the video frame begins at the flip-time.

16. The computer system of claim 15 , the method further comprising:
scanning the video frame at a scan-out rate that is higher than a frame rate used for generating a plurality of video frames when executing the video game; and
beginning to encode the video frame before the next occurrence of the server VSYNC signal.

17. The computer system of claim 15 , wherein in the method the performing the scan-out process for the video frame includes:
scanning the video frame scanline-by-scanline to an input frame buffer.

18. The computer system of claim 17 , the method further comprising:
beginning to scan the video frame to the encoder before fully receiving the video frame at the input frame buffer.

19. The computer system of claim 17 , the method further comprising:
performing at least one of the following on the video frame received at the input frame buffer to generate a modified video frame:
decompressing DCC compressed surface, or
resolution scaling, or
color space conversion, or
degamma, or
HDR expansion, or
gamut remap, or
LUT shaping, or
tone mapping, or
blending gamma; and

scanning the modified video frame to the encoder.

20. The computer system of claim 15 , the method further comprising:
beginning to encode the video frame before completing the scan-out process for the video frame.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Colenbrander Roelof Roderick
• Cerny Mark E

Classifications

• H04L65/61

  for supporting one-way streaming services, e.g. Internet radio · CPC title

• A63F13/537

  using indicators, e.g. showing the condition of a game character on screen · CPC title

• A63F13/358

  Adapting the game course according to the network or server load, e.g. for reducing latency due to different connection speeds between clients · CPC title

• A63F2300/538

  for performing operations on behalf of the game client, e.g. rendering · CPC title

• A63F2300/534

  for network load management, e.g. bandwidth optimization, latency reduction · CPC title