Processing of video operations as subtasks for video fragments

Patent metadata

Field	Value
Publication number	US-12363358-B2
Application number	US-202217843549-A
Country	US
Kind code	B2
Filing date	Jun 17, 2022
Priority date	Jun 17, 2022
Publication date	Jul 15, 2025
Grant date	Jul 15, 2025

Abstract

Techniques and solutions are described for executing a video processing task. A video processing task is received that includes one or more operations to be performed on a digital video file and an identifier of the digital video file. The video processing task is divided into subtasks of operations to be performed on fragments of the video, such as fragments having a particular duration. The duration can correspond to a duration used for video streaming Compared with video processing that is performed as a single task, disclosed techniques can provide improved fault tolerance, as only failed tasks need to be reprocessed. Video processing subtasks can be distributed to a plurality of workers, which can further improve fault tolerance, and can increase the computing power available for video processing, including allowing for the use of heterogenous or unreliable workers.

First claim

What is claimed is:

1. A computing system comprising:
at least one memory;
at least one hardware processor coupled to the at least one memory; and
one or more computer readable storage media storing computer-executable instructions, that when executed, cause the computing system to perform video processing operations that provide improved computing flexibility and fault tolerance, the operations comprising:
receiving a video processing task comprising an identifier of a digital video file and one or more operations to be performed on at least a portion of the digital video file, wherein the task specifies a fixed duration of video fragments to be produced as a result of the one or more operations;
generating a plurality of subtasks for processing the one or more operations, a given subtask of the plurality of subtasks identifying, or comprising, a fragment of the digital video file comprising multiple frames of the digital video file to be processed as part of the subtask, and the given subtask defining an output of a processed video fragment that has the fixed duration;
scheduling at least a first portion of the plurality of subtasks to one or more worker computing devices; and
sending subtasks of the at least a first portion of the plurality of subtasks to respective worker computing devices to be processed by the respective worker computing devices, thereby providing that output of the given subtasks is streamable without further processing of the processed video fragments to alter a duration of the output, avoiding additional coding steps, thus reducing computer resource use, reducing video processing time, or providing improved video quality.

2. The computing system of claim 1 , wherein the scheduling at least a first portion of the plurality of subtasks comprising scheduling the at least a first portion of the plurality of subtasks to a plurality of workers.

3. The computing system of claim 2 , wherein workers of the plurality of workers have heterogenous processing capabilities.

4. The computing system of claim 1 , wherein the scheduling at least a first portion of the plurality of subtasks comprises scheduling the at least a first portion of the plurality of subtasks to a single worker.

5. The computing system of claim 1 , the operations further comprising:
receiving a request to increase a priority of the video processing task; and
(1) reassigning a subtask of the plurality of subtasks to a more performant worker computing device than a worker computing device to which the subtask was originally assigned; or
(2) sending additional subtasks of the plurality of subtasks to additional worker computing devices.

6. The computing system of claim 1 , the operations further comprising:
receiving a request to decrease a priority of the video processing task; and
(1) delaying or cancelling execution of one or more subtasks of the plurality of subtasks; or
(2) reassigning a subtask of the plurality of subtasks to a less performant worker computing device than a worker computing device to which the subtask was originally assigned.

7. The computing system of claim 1 , the operations further comprising:
determining that a first subtask of the at least a portion of the plurality of subtasks has completed; and
sending a second subtask, not comprised within the at least a portion of the plurality of subtasks, to a worker computing device.

8. The computing system of claim 1 , wherein the sending the subtask comprises sending subtask information comprising the one or more operations, or instructions derived at least in part therefrom, and video data to be processed by the respective worker computing device for a respective subtask of the at least a first portion of the plurality of subtasks.

9. The computing system of claim 1 , wherein the sending the subtask comprises sending subtask information comprising the one or more operations, or instructions derived at least in part therefrom, and a time associated with a beginning of a respective fragment of the digital video file.

10. The computing system of claim 1 , wherein the sending the subtask comprises sending subtask information comprising the one or more operations, or instructions derived at least in part therefrom, and a frame associated with a beginning of a respective fragment of the digital video file.

11. The computing system of claim 1 , wherein the sending the subtask comprises sending subtask information comprising the one or more operations, or instructions derived at least in part therefrom, and a header for the digital video file.

12. The computing system of claim 1 , further operations further comprising:
receiving a request to stream at least one processed video fragment produced by a worker computing device from a subtask; and
sending the at least one processed video fragment to a streaming client in response to the request.

13. The computing system of claim 1 , the operations further comprising:
subsequent to the sending subtasks, receiving a request from a respective worker device requesting frames of the digital video file; and
sending frames of the digital video file to the respective worker device in response to the request.

14. The computing system of claim 13 , wherein the frames of the digital video file send to the respective worker device in response to the request are frames outside of the fixed duration of the subtask of the respective worker device.

15. The computing system of claim 1 , the operations further comprising:
subsequent to the sending subtasks, receiving a request from a respective worker device requesting metadata the digital video file; and
sending the metadata of the digital video file to the respective worker device in response to the request.

16. The computing system of claim 1 , wherein the metadata comprises a header of the digital video file.

17. A method that causes a computing system to perform video processing operations that provide improved computing flexibility and fault tolerance, the computing system comprising at least one memory and at least one hardware processor coupled to the at least one memory, the method comprising:
receiving a video processing task comprising an identifier of a digital video file and one or more operations to be performed on at least a portion of the digital video file, wherein the task specifies a fixed duration of video fragments to be produced as a result of the one or more operations;
generating a plurality of subtasks for processing the one or more operations, a given subtask of the plurality of subtasks identifying, or comprising, a fragment of the digital video file comprising multiple frames of the digital video file to be processed as part of the subtask, and the given subtask defining an output of a processed video fragment that has the fixed duration;
scheduling at least a first portion of the plurality of subtasks to one or more worker computing devices; and
sending subtasks of the at least a first portion of the plurality of subtasks to respective worker computing devices to be processed by the respective worker computing devices, thereby providing that output of the given subtasks is streamable without further processing of the processed video fragments to alter a duration of the output, avoiding additional coding steps, thus reducing computer resource use, reducing video processing time, or providing improved video quality.

18. The method of claim 17 , further comprising:
subsequent to the sending subtasks, receiving a request from a respective worker device requesting frames of the digit

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Heckman Nicholas Tegan
• Peterson Steven Craig
• Suri Nitin
• Whitehouse Jason Allen

Classifications

• G06F11/07

  Responding to the occurrence of a fault, e.g. fault tolerance · CPC title

• H04N19/48

  using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data (motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513) · CPC title

• G06F11/0793

  Remedial or corrective actions (recovery from an exception in an instruction pipeline G06F9/3861; by retry G06F11/1402; for recovering from a failure of a protocol instance or entity H04L69/40) · CPC title

• H04N21/234 Primary

  Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs · CPC title

• G06F9/5044 Primary

  considering hardware capabilities · CPC title