Rate adaptive encoding decoding scheme for prioritized segmented data

Patent metadata

Field	Value
Publication number	US-11438627-B2
Application number	US-202017131740-A
Country	US
Kind code	B2
Filing date	Dec 22, 2020
Priority date	Dec 22, 2020
Publication date	Sep 6, 2022
Grant date	Sep 6, 2022

Abstract

A method in a rate adaptive system includes categorizing, by an encoder, a plurality of clusters of data in a segmented image into one of a plurality of categories corresponding to a different predetermined label with a predetermined priority level; vectorizing, by the encoder, the data to generate a sparse vector xi for its corresponding label; encoding, by the encoder, a plurality of the sparse vectors xi by multiplying a measurement matrix Ai(t) with the sparse vector xi to generate a set of encoded information yi; transmitting, by the encoder to the decoder, the plurality of sets of the encoded information yi in a prioritized order; decoding, by the decoder, the plurality of sets of encoded information yi to determine the plurality of the sparse vectors xi based on determining measurement matrix Ai(t); and uniting the plurality of the sparse vectors xi into a single image frame.

First claim

What is claimed is:

1. A rate adaptive system for enabling perception sharing of a scene, the system comprising:
a priority-based encoder at a viewing system comprising a controller configured to:
split a plurality of clusters of data in a received segmented image of the scene according to one of a plurality of different predetermined labels wherein each label is assigned a predetermined priority level;
vectorize the data for a plurality of the labels to generate a sparse vector x i for its corresponding label that has the priority level of its corresponding label;
encode a plurality of the sparse vectors x i by multiplying a measurement matrix A i (t) with the sparse vector x i to generate a set of encoded information y i for the corresponding label that has the priority level of its corresponding label to generate a plurality of sets of the encoded information y i , wherein the measurement matrix A i (t) is configured to allow for data corresponding to higher priority level sparse vectors x i to be encoded with higher resolution than data corresponding to lower priority level sparse vectors x i ; and
transmit the plurality of sets of the encoded information y i in a prioritized order wherein higher priority level sets of the encoded information y i are transmitted prior to lower priority level sets of the encoded information y i ; and

a priority-based decoder at a remote system comprising a controller configured to:
receive the plurality of sets of encoded information y i ;
decode the plurality of sets of encoded information y i to determine a plurality of sparse vectors {circumflex over (x)} i based on determining measurement matrix A i (t); and
unite the plurality of sparse vectors {circumflex over (x)} i into a united vector;

wherein the united vector can be reshaped into an approximated image of the scene comprising an approximated image segmentation.

2. The rate adaptive system of claim 1 , wherein the priority-based encoder is further configured to encode the plurality of the sparse vectors x i in a prioritized order wherein higher priority level sparse vectors x i are encoded prior to lower priority level sparse vectors x i .

3. The rate adaptive system of claim 1 , wherein the priority-based decoder is further configured to decode the plurality of sets of encoded information y i to determine the plurality of sparse vectors {circumflex over (x)} i in a prioritized order wherein higher priority level sets of encoded information y i are decoded prior to lower priority level sets of encoded information y i .

4. The rate adaptive system of claim 1 , wherein the priority-based decoder is further configured to decode the plurality of sets of encoded information y i to determine the plurality of sparse vectors {circumflex over (x)} i based on y i =A i (t) x i such that ∥x i ∥ is minimal.

5. The rate adaptive system of claim 1 , wherein the measurement matrix A i (t) is determined at the priority-based encoder and at the priority-based decoder based on a common seed known at both the priority-based encoder and at the priority-based decoder.

6. The rate adaptive system of claim 1 , wherein dimensions of the measurement matrix A i (t) are determined at the priority-based encoder based on feedback provided by the priority-based decoder to the priority-based encoder.

7. The rate adaptive system of claim 1 , wherein both the prioritized encoder and the prioritized decoder are configured to determine dimensions of the measurement matrix A i (t) based on estimating a sparsity level s of a label according to a number of pixels segmented with the label.

8. The rate adaptive system of claim 7 , wherein both the prioritized encoder and the prioritized decoder are further configured to determine dimensions of the measurement matrix A i (t) to be used in a next frame based on the sparsity of each label in a current frame and subject to a sum of rates being less than or equal to a constraint.

9. The rate adaptive system of claim 1 , wherein:
the segmented image of the scene is derived from a camera image, RADAR depth image, LiDAR depth image, or sound image;
the viewing system comprises an autonomous or semi-autonomously driven vehicle; and
the remote system comprises another vehicle, a cloud-based computing system, infrastructure, or an edge-cloud-based computing system.

10. A method in a rate adaptive system comprising a priority-based encoder at a viewing system and a priority-based decoder at a remote system for enabling perception sharing of a scene, the method comprising:
splitting, by the priority-based encoder, a plurality of clusters of data in a received segmented image of the scene according to one of a plurality of different predetermined labels wherein each label is assigned a predetermined priority level;
vectorizing, by the priority-based encoder, the data for a plurality of the labels to generate a sparse vector x i for its corresponding label that has the priority level of its corresponding label;
encoding, by the priority-based encoder, a plurality of the sparse vectors x i by multiplying a measurement matrix A i (t) with the sparse vector x i to generate a set of encoded information y i for the corresponding label that has the priority level of its corresponding label to generate a plurality of sets of the encoded information y i , wherein the measurement matrix A i (t) is configured to allow for data corresponding to higher priority level sparse vectors x i to be encoded with higher resolution than data corresponding to lower priority level sparse vectors x i ;
transmitting, by the priority-based encoder to the priority-based decoder, the plurality of sets of the encoded information y i in a prioritized order, wherein higher priority level sets of the encoded information y i are transmitted prior to lower priority level sets of the encoded information y i ;
decoding, by the priority-based decoder, the plurality of sets of encoded information y i to determine a plurality of sparse vectors {circumflex over (x)} i based on determining measurement matrix A i (t);
uniting the plurality of sparse vectors {circumflex over (x)} i into a united vector, wherein the united vector can be reshaped into an approximated image of the scene comprising an approximated image segmentation.

11. The method of claim 10 , wherein the encoding the plurality of the sparse vectors x i comprises encoding the plurality of the sparse vectors x i in a prioritized order wherein higher priority level sparse vectors x i are encoded prior to lower priority level sparse vectors x i .

12. The method of claim 10 , wherein the decoding the plurality of sets of encoded information y i comprises decoding the plurality of sets of encoded information y i in a prioritized order wherein higher priority level sets of encoded information y i are decoded prior to lower priority level sets of encoded information y i .

13. The method of claim 10 , wherein the decoding the plurality of sets of encoded information y i comprises decoding the plurality of sets of encoded information y i to determine the plurality of sparse vectors {circumflex over (x)} i based on y i =A i (t) x i such that ∥x i ∥ is minimal.

14. The method of claim 10 , further comprising determining the measurement matrix A i (t) at the priority-based encoder and at the priority-based decoder based on a common seed known at both the priority-based encoder and at the priority-based decoder.

15. The method of claim 10 , further comprising determining dimensions of the measurement matrix A i (t) at the priority-based encoder and at the priority-base

Assignees

• Gm Global Tech Operations Llc

Inventors

• Kampeas Joseph
• Bustin Ronit
• Kishon Eran

Classifications

• H04L1/0009 Primary

  by adapting the channel coding (H04L1/1812 takes precedence) · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/30196

  Human being; Person · CPC title

• G06T7/11

  Region-based segmentation · CPC title