Adaptive screen encoding control

Patent metadata

Field	Value
Publication number	US-11277227-B2
Application number	US-202016798074-A
Country	US
Kind code	B2
Filing date	Feb 21, 2020
Priority date	Aug 19, 2013
Publication date	Mar 15, 2022
Grant date	Mar 15, 2022

Abstract

An adaptive screen encoding method comprising: using a computer, creating and storing, in computer memory, a plurality of conditions for use by a server configured to determine which of picture coding type to select; detecting a current picture by a sender for a content type including textual content, graphical content, and natural image content; determining a percentage of static macroblocks corresponding to the current picture; selecting the picture coding type based on the content type, the plurality of conditions, and the percentage of static macroblocks, wherein the method is performed by one or more special-purpose computing devices.

First claim

What is claimed is:

1. A method comprising:
encoding, in accordance with a plurality of adaptive bitrate control rules for processing a data stream including pictures associated with real-time screen share content presented during a video conference, each of the pictures as one of a long term reference (LTR) picture or a short term reference (STR) picture, wherein LTR pictures correspond to the real-time screen share content and the STR pictures correspond to cursor operations on screen while the real-time screen share content is presented; and
responsive to identifying one or more second pictures encoded as STR pictures that precede a first picture encoded as a LTR picture within a reference picture list for decoding the data stream, removing the one or more second pictures from the reference picture list.

2. The method of claim 1 , the method comprising:
detecting a complete plurality of pictures, wherein each of the complete plurality of pictures is encoded as one of a LTR picture or a STR picture.

3. The method of claim 2 , the method comprising:
identifying the first picture from the complete plurality of pictures encoded as an LTR picture.

4. The method of claim 3 , wherein identifying the first picture as a LTR picture includes identifying a first preceding picture which comes before the first picture in decoding order, wherein the first preceding picture is encoded as a STR picture.

5. The method of claim 4 , wherein removing the one or more second pictures includes removing the first preceding picture from the complete plurality of pictures and forming a reduced plurality of pictures, wherein the reduced plurality of pictures represents the complete plurality of pictures.

6. The method of claim 1 , the method comprising:
transmitting a complete plurality of pictures to a server.

7. The method of claim 6 , the method comprising:
receiving a reduced plurality of pictures from the server.

8. The method of claim 1 , the method comprising:
identifying a second preceding picture from the first picture, wherein the second preceding picture comes before a first preceding picture in decoding order and the second preceding picture is encoded as a STR picture.

9. The method of claim 8 , wherein identifying the second preceding picture from the first picture comprises:
determining the second preceding picture coming before the first preceding picture in decoding order, wherein the second preceding picture is encoded as a STR picture.

10. The method of claim 8 , the method comprising:
removing the second preceding picture from a complete plurality of pictures to form a reduced plurality of pictures.

11. A system, comprising:
means for encoding, in accordance with a plurality of adaptive bitrate control rules for processing a data stream including pictures associated with real-time screen share content presented during a video conference, each of the pictures as one of a long term reference (LTR) picture or a short term reference (STR) picture, wherein LTR pictures correspond to the real-time screen share content and the STR pictures correspond to cursor operations on screen while the real-time screen share content is presented; and
means for, responsive to identifying one or more second pictures encoded as STR pictures that precede a first picture encoded as a LTR picture within a reference picture list for decoding the data stream, removing the one or more second pictures from the
reference picture list.

12. The system of claim 11 , the system comprising:
means for receiving the reference picture list from a server.

13. The system of claim 11 , the system comprising:
means for identifying the first picture as a LTR picture by identifying a first preceding picture which comes before the first picture in decoding order, wherein the first preceding picture is encoded as a STR picture.

14. The system of claim 11 , the system comprising:
means for identifying a second preceding picture from the first picture, wherein the second preceding picture comes before a first preceding picture in decoding order and the second preceding picture is encoded as a STR picture;
means for determining the second preceding picture coming before the first preceding picture in decoding order, wherein the second preceding picture is encoded as a STR picture; and
means for removing the second preceding picture from a complete plurality of pictures to form a reduced plurality of pictures.

15. An apparatus, comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
encode, in accordance with a plurality of adaptive bitrate control rules for processing a data stream including pictures associated with real-time screen share content presented during a video conference, each of the pictures as one of a long term reference (LTR) picture or a short term reference (STR) picture, wherein LTR pictures correspond to the real-time screen share content and the STR pictures correspond to cursor operations on screen while the real-time screen share content is presented; and
responsive to an identification of one or more second pictures encoded as STR pictures that precede a first picture encoded as a LTR picture within a reference picture list for decoding the data stream, remove the one or more second pictures from the reference picture list.

16. The apparatus of claim 15 , wherein the first picture discontinues previous pictures immediately before the first picture.

17. The apparatus of claim 15 , wherein, to encode each of the pictures as one of a LTR picture or a STR picture, the processor is configured to execute the instructions to:
retrieve data indicative of the plurality of adaptive bitrate control rules from the memory.

18. The apparatus of claim 15 , wherein the processor is configured to execute the instructions to:
manage the reference picture list with a temporal identifier to include one or more of the pictures encoded as a LTR picture and one or more of the pictures encoded as a STR picture for each temporal layer of the data stream.

19. The apparatus of claim 15 , wherein the processor is configured to execute the instructions to:
identify the first picture as a LTR picture by identifying a first preceding picture which comes before the first picture in decoding order, wherein the first preceding picture is encoded as a STR picture.

20. The apparatus of claim 15 , wherein the processor is configured to execute the instructions to:
identify a second preceding picture from the first picture, wherein the second preceding picture comes before a first preceding picture in decoding order and the second preceding picture is encoded as a STR picture;
determine the second preceding picture coming before the first preceding picture in decoding order, wherein the second preceding picture is encoded as a STR picture; and
remove the second preceding picture from a complete plurality of pictures to form a reduced plurality of pictures.

Assignees

• Zoom Video Communications Inc

Inventors

• Wu Jing
• Jia Zhaofeng
• Ling Bo
• Liu Qiyong

Classifications

• H04L1/0009 Primary

  by adapting the channel coding (H04L1/1812 takes precedence) · CPC title

• H04L65/70

  Media network packetisation · CPC title

• H04L65/80

  Responding to QoS · CPC title

• H04L2001/0093

  Point-to-multipoint · CPC title

• H04N19/179

  the unit being a scene or a shot · CPC title