Selection of encoder and decoder for a video communications session

Patent metadata

Field	Value
Publication number	US-10045089-B2
Application number	US-201113196850-A
Country	US
Kind code	B2
Filing date	Aug 2, 2011
Priority date	Aug 2, 2011
Publication date	Aug 7, 2018
Grant date	Aug 7, 2018

Abstract

For a first device participating in a video communications session with a second device, some embodiments provide a method that receives sets of encoding features supported by each of one or more decoders of the second device. The method selects an encoder for the video communication session from several encoders available to the first device. The method selects a set of encoding features of the selected encoder based on the sets of encoding features received from the second device. The method transmits an encoded video bitstream to the second device along with information identifying the selected features in order for the second device to select a decoder to decode the bitstream.

First claim

We claim:

1. For a first device participating in a video communications session with a second device, a method comprising:
receiving sets of encoding features supported by each of one or more decoders of the second device;
selecting, while initiating the video communications session with the second device, (i) a first encoder for the video communications session from a plurality of encoders available to the first device, the plurality of encoders comprising at least one hardware encoder and at least one software encoder, and (ii) a set of encoding features of the selected first encoder, the selections based on the sets of encoding features received from the second device;
encoding a video bitstream in a particular encoding format using the selected set of encoding features of the first encoder;
transmitting an encoded video bitstream to the second device along with information identifying the selected features in order for the second device to select a decoder to decode the encoded video bitstream;
detecting a change in conditions at the first device; and
selecting, in response to the detected change, a second, separate encoder for encoding subsequent images of the video bitstream in the same particular encoding format using a different set of encoding features supported by the second encoder.

2. The method of claim 1 , wherein the method is performed by a video-conferencing application of the device.

3. The method of claim 1 , wherein the selections of the encoders and the sets of encoding features are further based on a combination of current resource usage at the first device, conditions of a network connection between the first device and the second device, and encoding features supported by the selected encoders.

4. The method of claim 1 , wherein at least one hardware encoder does not support a set of error resilient features and at least one software encoder supports the set of error resilient features, wherein when conditions of a network connection between the first device and the second device are better than a threshold level, selecting the first encoder comprises selecting the hardware encoder.

5. The method of claim 1 , wherein at least one hardware encoder does not support a set of error resilient features and at least one software encoder supports the set of error resilient features, wherein when conditions of a network connection between the first device and the second device are worse than a threshold level, selecting the first encoder comprises selecting the software encoder and selecting the set of encoding features comprises selecting the set of error resilient features.

6. The method of claim 1 , wherein the encoding features comprise at least one of video resolution adaptation, long-term reference frame, bandwidth probing, and adaptive sharpening.

7. The method of claim 1 , wherein selecting the set of encoding features comprises selecting a set of features supported by the selected encoder and by at least one decoder of the second device.

8. The method of claim 1 , wherein selecting the set of encoding features comprises:
determining all features supported by the selected first encoder;
identifying subsets of the features, each subset supported by at least one decoder of the second device; and
selecting one of the subsets as the set of encoding features.

9. The method of claim 8 , wherein one of the subsets comprises all features supported by the selected first encoder.

10. The method of claim 8 , wherein selecting one of the subsets comprises:
ranking the subsets; and
selecting the subset with the highest rank.

11. The method of claim 1 , wherein the features modify the encoded bitstream for a given encoding format.

12. The method of claim 11 , wherein the encoding format is H.264.

13. The method of claim 1 further comprising encoding a sequence of images captured at the first device as the encoded video bitstream using the selected encoder.

14. The method of claim 1 , wherein the software encoder supports a set of error-resilient features and the hardware encoder does not support the set of error-resilient features, wherein the detected change is an increase of packets being dropped from the transmitted encoded video bitstream, and selecting a second encoder comprises switching from the hardware encoder to the software encoder in order to use the set of error-resilient features.

15. A non-transitory machine readable medium of a first device, the machine readable medium storing a video conferencing application which when executed by at least one processing unit of the first device participates in a videoconference with a second device, the video conferencing application comprising sets of instructions for:
transmitting, to the second device, sets of encoding features supported by each of a plurality of decoders of the first device, the plurality of decoders comprising at least one hardware decoder and at least one software decoder that both decode bitstreams encoded in a same particular encoding format, wherein the second device uses the sets of encoding features to select an encoder from a plurality of encoders while initiating the videoconference with the first device;
receiving an encoded video bitstream from the second device along with a set of features used to encode the bitstream at the second device in the particular encoding format;
based on the set of features used to encode the bitstream, selecting a first decoder from the plurality of decoders at the first device for decoding the bitstream from the second device by selecting a highest priority decoder of the plurality of decoders that supports all features used to encode the bitstream; and
in response to receiving a modified bitstream during the video conference, selecting a second, separate decoder of the plurality of decoders that supports all of the features used to encode the modified bitstream.

16. The non-transitory machine readable medium of claim 15 , wherein the set of instructions for selecting a decoder comprises a set of instructions for ranking the decoders in order of priority.

17. The non-transitory machine readable medium of claim 16 , wherein the decoder ranking is based on current resource usage at the first device.

18. The non-transitory machine readable medium of claim 15 , wherein the hardware decoder has a higher priority than the software decoder.

19. The non-transitory machine readable medium of claim 15 , wherein the video conferencing application further comprises sets of instructions for:
receiving sets of encoding features supported by each of one or more decoders of the second device;
selecting (i) an encoder for the videoconference from a plurality of encoders available to the first device and (ii) a set of encoding features of the selected encoder based on the sets of encoding features supported by the decoders of the second device; and
transmitting an encoded video bitstream to the second device.

20. The non-transitory machine readable medium of claim 15 , wherein the set of instructions for transmitting the sets of encoding features comprises a set of instructions for transmitting a feature string that lists the encoding features of a particular encoder and specifies how the second device should indicate to the first device which of the encoding features are used to generate the encoded video bitstream.

21. The non-transitory machine readable medium of claim 15 , wherein the set of instructions for selecting a second, separate decoder comprises sets of instructions

Assignees

• Jeong Hyeonkuk
• Garcia Jr Roberto
• Zhou Xiaosong
• Price Douglas S
• Abuan Joe S
• Apple Inc

Inventors

• Jeong Hyeonkuk
• Garcia Jr Roberto
• Zhou Xiaosong
• Price Douglas S
• Abuan Joe S

Classifications

• H04N21/6379

  directed to encoder {, e.g. for requesting a lower encoding rate} · CPC title

• H04N21/65

  Transmission of management data between client and server · CPC title

• H04N21/4788

  communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)} · CPC title

• H04N19/156

  Availability of hardware or computational resources, e.g. encoding based on power-saving criteria · CPC title

• H04N21/6547 Primary

  comprising parameters, e.g. for client setup · CPC title