Overlapping rendering, streamout, and display at a client of rendered slices of a video frame

Patent metadata

Field	Value
Publication number	US-12415129-B2
Application number	US-202318516867-A
Country	US
Kind code	B2
Filing date	Nov 21, 2023
Priority date	Oct 1, 2019
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

A method of cloud gaming is disclosed. The method including receiving an encoded video frame at a client, wherein a server executes an application to generate a rendered video frame which is then encoded at an encoder at the server as the encoded video frame, wherein the encoded video frame includes one or more encoded slices that are compressed. The method including decoding the one or more encoded slices at a decoder of the client to generate one or more decoded slices. The method including rendering the one or more decoded slices for display at the client. The method including begin displaying the one or more decoded slices that are rendered before fully receiving the one or more encoded slices at the client.

First claim

What is claimed is:

1. A method, comprising:
begin receiving from a server a plurality of encoded slices of an encoded video frame at a client device;
decoding an encoded slice that is received to generate a decoded slice; and
begin rendering the decoded slice for presentation on a display associated with the client device before fully receiving each of the plurality of encoded slices of the encoded video frame,
wherein the rendering the decoded slice includes formatting the decoded slice as at least a portion of a screen slice for presentation on the display,
wherein the screen slice corresponds to a scanline of the display.

2. The method of claim 1 , wherein the rendering the decoded slice further includes:
beginning the rendering of the decoded slice before fully decoding each of the plurality of encoded slices of the encoded video frame.

3. The method of claim 1 , further comprising:
formatting the decoded slice as at least a portion of one or more screen slices.

4. The method of claim 1 , further comprising:
storing the at least a portion of the screen slice in a display buffer; and
streaming out from the display buffer to the display the at least a portion of the screen slice before fully receiving each of the plurality of encoded slices of the encoded video frame.

5. The method of claim 1 , further comprising:
storing the at least a portion of the screen slice in a display buffer; and
streaming out from the display buffer to the display the at least a portion of the screen slice before fully decoding each of the plurality of encoded slices of the encoded video frame.

6. The method of claim 1 , further comprising:
storing the at least a portion of the screen slice in a display buffer; and
streaming out from the display buffer to the display the at least a portion of the screen slice before fully rendering each of the plurality of encoded slices.

7. The method of claim 1 , further comprising:
decoding the plurality of encoded slices of the video frame to generate a plurality of decoded slices; and
streaming out from a display buffer to the display the at least a portion of the screen slice before fully rendering each of the plurality of decoded slices.

8. The method of claim 1 ,
wherein the rendering of the decoded slice begins before a video frame is fully scanned-out to an encoder at a server,
wherein the encoder encodes the video frame as the plurality of encoded slices,
wherein the server executes an application to generate the video frame.

9. A computer system comprising:
a processor;
memory coupled to the processor and having stored therein instructions that, if executed by the computer system, cause the computer system to execute a method, comprising:
begin receiving from a server a plurality of encoded slices of an encoded video frame at a client device;
decoding an encoded slice that is received to generate a decoded slice; and
begin rendering the decoded slice for presentation on a display associated with the client device before fully receiving each of the plurality of encoded slices of the encoded video frame,
wherein the rendering the decoded slice includes formatting the decoded slice as at least a portion of a screen slice for presentation on the display,
wherein the screen slice corresponds to a scanline of the display.

10. The computer system of claim 9 , wherein in the method the rendering the decoded slice further includes:
beginning the rendering of the decoded slice before fully decoding each of the plurality of encoded slices of the encoded video frame.

11. The computer system of claim 9 , the method further comprising:
formatting the decoded slice as at least a portion of one or more screen slices.

12. The computer system of claim 9 , the method further comprising:
storing the at least a portion of the screen slice in a display buffer; and
streaming out from the display buffer to the display the at least a portion of the screen slice before fully receiving each of the plurality of encoded slices of the encoded video frame.

13. The computer system of claim 9 , the method further comprising:
storing the at least a portion of the screen slice in a display buffer; and
streaming out from the display buffer to the display the at least a portion of the screen slice before fully decoding each of the plurality of encoded slices of the encoded video frame.

14. The computer system of claim 9 , the method further comprising:
storing the at least a portion of the screen slice in a display buffer; and
streaming out from the display buffer to the display the at least a portion of the screen slice before fully rendering each of the plurality of encoded slices.

15. The computer system of claim 9 , the method further comprising:
decoding the plurality of encoded slices of the video frame to generate a plurality of decoded slices; and
streaming out from a display buffer to the display the at least a portion of the screen slice before fully rendering each of the plurality of decoded slices.

16. The computer system of claim 9 ,
wherein the rendering of the decoded slice begins before a video frame is fully scanned-out to an encoder at a server,
wherein the encoder encodes the video frame as the plurality of encoded slices,
wherein the server executes an application to generate the video frame.

17. A non-transitory computer-readable medium storing a computer program for performing a method, the computer-readable medium comprising:
program instructions to begin receiving from a server a plurality of encoded slices of an encoded video frame at a client device;
program instructions for decoding an encoded slice that is received to generate a decoded slice; and
program instructions to begin rendering the decoded slice for presentation on a display associated with the client device before fully receiving each of the plurality of encoded slices of the encoded video frame,
wherein the rendering the decoded slice includes formatting the decoded slice as at least a portion of a screen slice for presentation on the display,
wherein the screen slice corresponds to a scanline of the display.

18. The non-transitory computer-readable medium of claim 17 , wherein the program instructions for rendering the decoded slice includes:
program instructions for beginning the rendering of the decoded slice before fully decoding each of the plurality of encoded slices of the encoded video frame.

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Cerny Mark E
• Yong Kelvin M

Classifications

• H04L67/131

  Protocols for games, networked simulations or virtual reality · CPC title

• H04L65/70

  Media network packetisation · CPC title

• H04N21/4781

  Games · CPC title

• H04N21/2343

  involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements {(video transcoding H04N19/40; media packet handling at the source H04L65/762)} · CPC title

• G07F17/3225

  Data transfer within a gaming system, e.g. data sent between gaming machines and users · CPC title