Dynamic low latency mode for a digital video production system

Patent metadata

Field	Value
Publication number	US-11784839-B2
Application number	US-201916731869-A
Country	US
Kind code	B2
Filing date	Dec 31, 2019
Priority date	Dec 31, 2019
Publication date	Oct 10, 2023
Grant date	Oct 10, 2023

Abstract

Systems, devices and processes and described herein to improve the generation of video content in video production systems. In general, the systems, devices and processes facilitate the incorporation of one or more remotely captured video streams into an output video stream for use in video conferencing. Specifically, the systems, devices and processes provide for the use of a dynamically adjustable time clock delay to generate a relatively low latency output video stream suitable for use in video conferencing. The use of dynamically adjustable time clock delay allows one or more input video streams to be generated by one or more remote video capture devices, transmitted to a video access point device, and selectively combined at the video access point device into the output video stream with relatively low latency suitable for video conferencing.

First claim

What is claimed is:

1. An automated process executable by a video access point device that communicates with a plurality of video capture devices via a network, the automated process comprising:
generating a shared timestamp with a system time clock of the video access point device;
transmitting the shared timestamp from the video access point device to each of the plurality of video capture devices via the network;
subsequently receiving a plurality of input video streams by the video access point device, wherein each input video stream is received via the network from a separate one of the plurality of video capture devices, and wherein each of the input video streams is encoded by one of the plurality of video capture devices using the shared timestamp from the video access point device so that each of the plurality of input video streams is encoded with the same shared timestamp;
calculating, for each of the input video streams received from the video capture devices, a video latency based upon a difference between the shared timestamp that is encoded with packets of the input video stream and a current timestamp generated by the system time clock, wherein the video latency includes a first time to transmit the shared timestamp from the video access point device and a second time to receive from the video capture devices the input video streams encoded with the shared timestamp;
determining a maximum latency from the video latencies calculated for each of the input video streams;
encoding a single, combined conferencing video by the video access point device by compositing imagery from the input video streams having time latencies that are less than a maximum time clock delay parameter into the single, combined conferencing video stream while excluding video frames in the received input video streams having time latencies greater than the maximum time clock delay parameter from the conferencing video stream, wherein the maximum time clock delay parameter is based on the maximum latency; and
transmitting the single, combined conferencing video stream to the network for video conferencing.

2. The automated process of claim 1 wherein the maximum time clock delay parameter is equal to the maximum latency plus a margin value.

3. The automated process of claim 1 wherein the maximum time clock delay parameter is further adapted based on a selected latency mode from a plurality of latency modes.

4. The automated process of claim 1 wherein the video latencies for each of the input video streams are calculated based upon only a predetermined percentage of the packets arriving at the video access point device for that input video stream.

5. A video access point device comprising a processor and a memory, wherein the processor is configured to perform an automated process by executing digital instructions stored in the memory, wherein the automated process comprises:
generating a shared timestamp with a system time clock of the video access point device;
transmitting the shared timestamp from the video access point device to a first video capture device and a second video capture device;
receiving, at the video access point device, a first input video stream from the first video capture device at a first current time of the system time clock of the video access point device,
wherein the first input video stream is encoded by the first video capture device using the shared timestamp from the video access point device;

receiving a second video input stream from the second video capture device at a second current time of the system time clock of the video access point device,
wherein the second input video stream is encoded by the second video capture device using the shared timestamp from the video access point device,

calculating a first video latency of the first input video stream based on a difference between the shared timestamp and the first current time of the system time clock of the video access point device,
wherein the first video latency includes a first time to transmit the shared timestamp from the video access point device to the first video capture device and a second time to receive the first input video stream encoded with the shared timestamp from the first video capture device;

calculating a second video latency of the second input video stream based on a difference between the shared timestamp and the second current time of the system time clock of the video access point device,
wherein the second video latency includes a third time to transmit the shared timestamp from the video access point device to the second video capture device and a fourth time to receive the second input video stream encoded with the shared timestamp from the second video capture device;

determining a maximum latency from the first video latency and the second video latency;
encoding, by the video access point device, a single video stream by compositing imagery from the first input video stream and the second video input stream while excluding video frames having time latencies greater than a maximum time clock delay parameter; and
transmitting the single video stream to a network for video conferencing.

6. The video access point device of claim 5 , wherein the first video latency of the first input video stream is calculated based upon a predetermined percentage of packets of the first input video stream arriving at the video access point device.

7. The video access point device of claim 5 wherein the encoding comprises applying the maximum time clock delay parameter to a system time clock on the video access point device.

8. The video access point device of claim 5 wherein the maximum time clock delay parameter is based at least in part upon a selected latency mode from a plurality of latency modes.

Assignees

• Sling Media Pvt Ltd
• Dish Network Technologies India Pvt Ltd

Inventors

• Sharma S A Varun Hrishikesh

Classifications

• H04L12/1827 Primary

  Network arrangements for conference optimisation or adaptation · CPC title

• H04N21/4223 Primary

  Cameras (H04N23/00 takes precedence) · CPC title

• H04L12/189

  in combination with wireless systems (selective distribution or broadcast in wireless communication networks H04W4/06) · CPC title

• H04L12/1831

  Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status · CPC title

• H04L65/70

  Media network packetisation · CPC title