Seamless video pipeline transition between WiFi and cellular connections for real-time applications on mobile devices

Patent metadata

Field	Value
Publication number	US-9553913-B2
Application number	US-201414338734-A
Country	US
Kind code	B2
Filing date	Jul 23, 2014
Priority date	May 30, 2014
Publication date	Jan 24, 2017
Grant date	Jan 24, 2017

Abstract

Performing a real-time application on a mobile device, involving communication of audio/video packets with a remote device. The mobile device may initially communicate the audio/video packets on a first communication channel with the remote device. During the real-time communication, the mobile device may determine if no packets have been received by the mobile device from the remote device for a first threshold period of time. If no packets have been received by the mobile device from the remote device for the first threshold period of time, the mobile device may establish a second communication channel for transmission of the audio/video packets with the remote device. In response to using the second communication channel, the mobile device may modify a resolution or bit rate of the audio/video packets transmitted to the remote device.

First claim

What is claimed is:

1. A method for conducting a real time application on a mobile device, the method comprising:
at the mobile device:
capturing first images from a camera of the mobile device using first camera capture settings having a first resolution;
communicating first multimedia packets on a first communication channel during a video session, wherein the first multimedia packets correspond to the first images captured from the camera using the first camera capture settings having the first resolution, wherein the first multimedia packets comprise a first number of video frames, wherein the first number of video frames includes non-droppable frames and droppable frames;
in response to transitioning to communication of multimedia packets on a second communication channel during the video session, changing the camera capture settings of the camera to second camera capture settings having a second resolution, wherein the second communication channel is a different bandwidth channel than the first communication channel, wherein the second resolution is different than the first resolution; and
communicating second multimedia packets on the second communication channel during the video session, wherein the second multimedia packets correspond to the second images captured from the camera using the second camera capture settings having the second resolution, wherein the second multimedia packets comprise a second number of video frames, wherein the second number of video frames includes non-droppable frames and a lesser number of droppable frames than when communicating on the first communication channel, wherein communication of the second multimedia packets on the second communication channel is performed concurrently with communication of the first multimedia packets on the first communication channel.

2. The method of claim 1 , wherein the mobile device communicates with a remote device, the method further comprising:
determining that the remote device has transitioned from generating multimedia packets on the first communication channel to generating multimedia packets on the second communication channel during the video session;
in response to said determining that the remote device has transitioned during the video session, changing the camera capture settings of the camera to the second camera capture settings having the second resolution.

3. The method of claim 1 , wherein the mobile device uses an encoder, the method further comprising:
in response to said changing the camera capture settings of the camera to the second camera capture settings having the second resolution, the mobile device modifying the encoder with different settings based on the changed camera settings.

4. The method of claim 3 , wherein the mobile device communicates with a remote device; the method further comprising:
after said modifying the encoder with different settings based on the changed resolution, transmitting, by the mobile device, an intra frame on the second communication channel in response to said transitioning, wherein the intra frame transmitted by the modified encoder comprises metadata useable by the remote device to change settings of a video decoder of the remote device.

5. The method of claim 4 , further comprising:
determining that the intra frame transmitted on the second communication channel in response to said transitioning was not received by the remote device;
retransmitting the intra frame on the second communication channel in response to said determining that the intra frame was not received by the remote device.

6. The method of claim 4 , further comprising:
during a transition period of said modifying the encoder with different settings, ignoring refresh frame requests until the intra frame transmitted on the second communication channel is determined to have been received by the remote device.

7. The method of claim 3 , further comprising:
during said communicating the first multimedia packets on the first communication channel, maintaining packets in a first queue usable by the video encoder;
in response to the mobile device modifying the encoder with different settings, flushing the packets from the first queue.

8. The method of claim 1 , wherein the first communication channel is a higher bandwidth channel than the second communication channel, wherein the first resolution is a higher resolution than the second resolution.

9. The method of claim 8 , wherein the first communication channel is Wireless LAN (WLAN), and wherein the second communication channel is cellular.

10. The method of claim 1 , wherein changing the camera settings is performed in an asymmetric manner, with respect to a remote device.

11. A mobile device configured to perform a real-time application, wherein the real-time application communicates audio/video packets with a remote device, the mobile device comprising:
a camera configured to capture images for the real-time application;
at least one antenna;
at least one radio, wherein the at least one radio is configured to perform communication a first channel and a second channel; and
one or more processors coupled to the at least one radio, wherein the one or more processors and the at least one radio are configured to:
communicate first multimedia packets on a first communication channel during a video session, wherein the first multimedia packets use first video settings associated with the first communication channel, wherein when the mobile device is communicating on the first communication channel, the mobile device is configured to transmit a first number of video frames on the first communication channel, wherein the first number of video frames includes non-droppable frames and droppable frames;
in response to transitioning to communication of multimedia packets on a second communication channel during the video session, change the first video settings to second video settings for the second communication channel, wherein the second communication channel is a different bandwidth channel than the first communication channel; and
communicate second multimedia packets on the second communication channel during the video session, wherein the second multimedia packets use the second video settings associated with the second communication channel, wherein the second multimedia packets comprise a second number of video frames, wherein the second number of video frames includes non-droppable frames and a lesser number of droppable frames than when communicating on the first communication channel, wherein communication of the second multimedia packets on the second communication channel is performed concurrently with communication of the first multimedia packets on the first communication channel.

12. The mobile device of claim 11 , wherein the mobile device uses an encoder for the multimedia packets, wherein one or more processors are further configured to:
modify the encoder with the first or second video settings based on whether the mobile device is currently communicating on the first or second communication channel.

13. The mobile device of claim 12 , wherein the first communication channel is a higher bandwidth channel than the second communication channel, wherein the video encoder is configured with first settings for generation of higher resolution images when the mobile device is communicating on the first communication channel, and wherein the video encoder is configured with second settings for generation of lower resolution images when the mobile device is communicating on the second communication channel.

14. The mobile device of claim 11 , wherein the first commu

Assignees

• Apple Inc

Inventors

• Yang Yan
• Zhou Xiasong
• Jeong Hyeonkuk
• Abuan Joe S
• Johar Gobind
• Shen Yichao

Classifications

• H04L65/1083

  In-session procedures · CPC title

• H04L65/80 Primary

  Responding to QoS · CPC title

• H04N7/147

  Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q) · CPC title

• H04W84/045

  using private Base Stations, e.g. femto Base Stations, home Node B · CPC title

• H04W36/1446

  wherein at least one of the networks is unlicensed · CPC title