Optimizing media experience in conferencing with diverse participants

Patent metadata

Field	Value
Publication number	US-12200401-B2
Application number	US-202217661947-A
Country	US
Kind code	B2
Filing date	May 4, 2022
Priority date	Jun 4, 2021
Publication date	Jan 14, 2025
Grant date	Jan 14, 2025

Abstract

Participant systems in an audiovisual (AV) conferencing can, in one embodiment, use methods to ensure that all participants have a common platform to support AV conferencing, such as a common codec, and also attempt to provide higher quality AV conferencing (e.g., better than the common codec) based on available bandwidth and other criteria. The participant systems can use the common codec as a fallback platform when bandwidth or other criteria dictate a reduction in the quality of the AV conferencing. Other embodiments are also disclosed.

First claim

What is claimed is:

1. A non-transitory machine readable medium storing executable program instructions which when executed by a first data processing system cause the first data processing system to perform a method in an environment that includes the first data processing system and a second data processing system and a third data processing system and a server, the method comprising:
transmitting, by the first data processing system to the server, codec capabilities of the first data processing system, the codec capabilities of the first data processing system indicating a first plurality of codecs in the first data processing system that are available for use in an audiovisual (AV) conference, wherein each codec in the first plurality of codecs and in the second plurality of codecs is configured to compress video content in an AV conference for transmission to other participants in the AV conference and is configured to decompress video content received in the AV conference;
receiving, by the first data processing system, a second plurality of codec capabilities of the second data processing system, the second set of codec capabilities indicating a second set of codecs in the second data processing system that are available for use in the AV conference;
receiving, at the first data processing system, a request from the second data processing system to receive a first stream, encoded with a first codec, in the AV conference, the first codec in the first plurality of codecs in the first data processing system;
transmitting, by the first data processing system to the server, the first stream for delivery to the second data processing system;
receiving, by the first data processing system, a request from the third data processing system for a second stream, encoded with a second codec, in the AV conference, the second codec being in the first plurality of codecs and being different than the first codec, wherein the first codec is available for use in the first and second data processing systems but not available for the third data processing systems, the second codec is a codec that is common to and available for use in the first, the second and the third data processing systems, and the second codec is a common codec for usage among the first, second, and third data processing systems when a fallback condition exists;
creating, on demand and in response to the request from the third data processing system, the second stream in the AV conference; and
transmitting the second stream to the server for delivery to the third data processing system while the first data processing system continues to transmit the first stream to the server for delivery to the second data processing system.

2. The non-transitory machine readable medium as in claim 1 wherein the first and the second data processing systems execute one or more versions of a first operating system during the AV conference and the third data processing system executes a second operating system during the AV conference.

3. The non-transitory machine readable medium as in claim 1 , wherein the method further comprises:
receiving, at the first data processing system, a third stream in the AV conference from the second data processing system, the third stream encoded with the first codec; decoding the third stream using the first codec; receiving, at the first data processing system, a fourth stream in the AV conference from the third data processing system, the fourth stream encoded with the second codec; decoding the fourth stream using the second codec at the first data processing system; and displaying video from the third stream, video from the fourth stream and video from the first stream on a display of the first data processing system.

4. The non-transitory machine readable medium as in claim 1 , wherein the method further comprises:
monitoring, by the first data processing system, one or more of: uplink bandwidth from the first data processing system, thermal data about a thermal state of the first data processing system, or battery state about a battery in the first data processing system;
in response to one or more conditions determined from the monitoring, switching from transmitting to the second data processing system the first stream to transmitting to the second data processing system the second stream.

5. The non-transitory machine readable medium as in claim 4 , wherein the switching comprises: associating the second stream with a stream identifier of the first stream to cause the server to forward the second stream to the second data processing system.

6. The non-transitory machine readable medium as in claim 4 , wherein the method further comprises:
transmitting, by the first data processing system, an indication to the server, for delivery to the second data processing system, that the first data processing system has or will stop transmitting the first stream for delivery to the second data processing system;
receiving a request, from the second data processing system, for the second stream;
transmitting, by the first data processing system, the second stream to the server for delivery to the second data processing system.

7. A method performed by a server in an environment that includes the server, a first data processing system and a second data processing system and a third data processing system, the method comprising:
receiving, by the server from the first data processing system, codec capabilities of the first data processing system, the codec capabilities of the first data processing system indicating a first plurality of codecs in the first data processing system that are available for use in an audiovisual (AV) conference, wherein each codec in the first plurality of codecs and in the second plurality of codecs is configured to compress video content in an AV conference for transmission to other participants in the AV conference and is configured to decompress video content received in the AV conference;
transmitting, by the server to the first data processing system, a second plurality of codec capabilities of the second data processing system, the second plurality of codec capabilities indicating a second plurality of codecs in the second data processing system that are available for use in the AV conference;
transmitting, by the server to the first data processing system, a request from the second data processing system to receive a first stream, encoded with a first codec, in the AV conference, the first codec in the first plurality of codecs in the first data processing system;
receiving, by the server from the first data processing system, the first stream for delivery to the second data processing system;
transmitting, by the server to first data processing system, a request from the third data processing system for a second stream, encoded with a second codec, in the AV conference, the second codec being in the first set of codecs and being different than the first codec, wherein the first codec is available for use in the first and second data processing systems but not available for the third data processing systems, the second codec is a codec that is common to and available for use in the first, the second and the third data processing systems, and the second codec is a common codec for usage among the first, second, and third data processing systems when a fallback condition exists; and
receiving the second stream at the server for delivery to the third data processing system while the server continues to transmit the first stream to the second data processing system.

8. The method as in claim 7 , wherein the method further comprises:
switching from transmitting to the second data processing system the first stream to transmitting to the second data pr

Assignees

• Apple Inc

Inventors

• Chandok Gurtej Singh G
• Garrido Christopher M
• Lu Chieh
• Pollack Daniel B
• Santhanam Karthick
• Biderman David L
• Qui Jinbo
• Zhang Dazhong
• Hinojosa Jose A Lozano
• Kobzar Igor

Classifications

• H04L65/60

  Network streaming of media packets · CPC title

• H04N19/40

  using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream · CPC title

• H04N7/15 Primary

  Conference systems · CPC title

• H04L65/403

  Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) · CPC title

• H04N7/152 Primary

  Multipoint control units therefor · CPC title