Rate adaptation for video coding for machines

Patent metadata

Field	Value
Publication number	US-12526439-B2
Application number	US-202418639731-A
Country	US
Kind code	B2
Filing date	Apr 18, 2024
Priority date	Apr 22, 2023
Publication date	Jan 13, 2026
Grant date	Jan 13, 2026

Abstract

An example a first network entity for processing feature set data formed from media data includes a processing system comprising one or more processors implemented in circuitry, the processing system being configured to: determine a first set of processing tasks of a series of processing tasks to be performed on a set of media data, the first set of processing tasks corresponding to tasks to be performed by the first network entity, wherein a second set of processing tasks is to be performed by a second network entity; perform the first set of processing tasks on the set of media data to form a feature map; encode the feature map to form an encoded feature map; and send the encoded feature map to the second network entity to enable the second network entity to perform the second set of processing tasks using the feature map.

First claim

What is claimed is:

1 . A method of processing feature set data formed from media data, the method comprising:
determining, by a first network entity, a first set of processing tasks of a series of processing tasks to be performed on a set of media data, wherein the series of processing tasks are associated with a task network configured to process a set of media data to perform at least one of instance segmentation or object tracking and wherein the first set of processing tasks correspond to tasks to be performed by the first network entity and a second set of processing tasks of the series of processing tasks is to be performed by a second network entity; receiving data representative of at least one identifier of one or more droppable tasks in the series of processing tasks, wherein performance of the one or more droppable tasks may be excluded during processing of the set of media data to perform the at least one of instance segmentation or object tracking; performing, by the first network entity, the first set of processing tasks on the set of media data to form a feature map indicative of at least one feature extracted from the set of media data, wherein the first network entity does not perform the one or more droppable tasks during performance of the first set of processing tasks; encoding, by the first network entity, the feature map to form an encoded feature map; and sending, by the first network entity, the encoded feature map to the second network entity to enable the second network entity to perform the second set of processing tasks using the feature map.

2 . The method of claim 1 , wherein the first network entity comprises a user equipment (UE) and the second network entity comprises an application server (AS).

3 . The method of claim 1 , wherein determining the first set of processing tasks comprises:
receiving, from the second network entity, supported configurations for the series of processing tasks; and determining a split point between the first set of processing tasks and the second set of processing tasks.

4 . The method of claim 3 , wherein the supported configurations include data representing a compression method for the feature map following one of the tasks of the series of processing tasks.

5 . The method of claim 4 , wherein the supported configurations further include data representing a parameter associated with the compression method.

6 . The method of claim 5 , wherein the parameter represents an amount of quantization applied to values of the feature map.

7 . The method of claim 1 , wherein determining the first set of processing tasks comprises:
determining a charge level of a battery of the first network entity; and determining the first set of processing tasks according to whether the charge level is above a threshold.

8 . The method of claim 1 , wherein determining the first set of processing tasks comprises:
determining an available amount of network bandwidth in a network by which the first network entity communicates with the second network entity; and determining the first set of processing tasks according to whether the available amount of network bandwidth is above a threshold.

9 . The method of claim 1 , further comprising sending data representative of the second set of processing tasks to the second network entity.

10 . The method of claim 9 , wherein sending the data representative of the second set of processing tasks comprises sending the data representative of the second set of processing tasks in at least one of a session description protocol (SDP) offer message, an SDP answer message, an encoder/decoder (CODEC) level message, a supplemental enhancement information (SEI) message, a real-time transport protocol (RTP) message, an HTTP message, an RTP control protocol (RTCP) message, or a secure RTP (SRTP) message.

11 . The method of claim 1 , further comprising receiving data representative of identifiers of two or more potential split points in the series of processing tasks, the split points each representing a point at which the series of processing tasks can be partitioned into the first set of processing tasks and the second set of processing tasks.

12 . The method of claim 11 , wherein the series of processing tasks is defined in a computing graph, and wherein the identifiers comprise labels generated by a computing graph description tool applied to the computing graph.

13 . The method of claim 12 , wherein the labels generated by the computing graph description tool conform to one of open neural network exchange (ONNX), neural network exchange formation (NNEF), uniform resource locators (URLs), uniform resource identifiers (URIs), or uniform resource names (URNs).

14 . The method of claim 11 , wherein the data representative of the identifiers of the two or more potential split points is separate from the data representative of the identifiers of the one or more droppable tasks.

15 . The method of claim 11 , wherein the data representative of the identifiers comprises encoded data representative of the identifiers.

16 . The method of claim 11 , wherein receiving the data representative of the identifiers comprises receiving the data representative of the identifiers from a task network repository.

17 . A first network entity for processing feature set data formed from media data, the first network entity comprising:
a memory configured to store media data; and a processing system comprising one or more processors implemented in circuitry, the processing system being configured to:
determine a first set of processing tasks of a series of processing tasks to be performed on a set of media data, wherein the series of processing tasks are associated with a task network configured to process a set of media data to perform at least one of instance segmentation or object tracking and wherein the first set of processing tasks correspond to tasks to be performed by the first network entity and a second set of processing tasks of the series of processing tasks is to be performed by a second network entity;
receive data representative of at least one identifier of one or more droppable tasks in the series of processing tasks, wherein performance of the one or more droppable tasks may be excluded during processing of the set of media data to perform the at least one of instance segmentation or object tracking;
perform the first set of processing tasks on the set of media data to form a feature map indicative of at least one feature extracted from the set of media data, wherein the first network entity does not perform the one or more droppable tasks during performance of the first set of processing tasks;
encode the feature map to form an encoded feature map; and
send the encoded feature map to the second network entity to enable the second network entity to perform the second set of processing tasks using the feature map.

18 . The first network entity of claim 17 , wherein the first network entity comprises a user equipment (UE) and the second network entity comprises an application server (AS).

19 . The first network entity of claim 17 , wherein to determine the first set of processing tasks, the processing system is configured to:
receive, from the second network entity, supported configurations for the series of processing tasks; and determine a split point between the first set of processing tasks and the second set of processing tasks.

20 . The first network entity of claim 19 , wherein the supported conf

Assignees

• Qualcomm Inc

Inventors

• Ma Liangping
• Leung Nikolai Konrad
• Bouazizi Imed
• Rico Alvarino Alberto
• Lo Charles Nung
• Stockhammer Thomas

Classifications

• H04N19/124 Primary

  Quantisation · CPC title

• H04N19/42 Primary

  characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation (H04N19/635 takes precedence) · CPC title