Methods and apparatus to process video frame pixel data using artificial intelligence video frame segmentation

Patent metadata

Field	Value
Publication number	US-12483701-B2
Application number	US-202117555119-A
Country	US
Kind code	B2
Filing date	Dec 17, 2021
Priority date	Dec 17, 2021
Publication date	Nov 25, 2025
Grant date	Nov 25, 2025

Abstract

Disclosed examples include video frame segmenter circuitry to generate segmentation data of first video frame pixel data, the segmentation data including metadata corresponding to a foreground region and a background region, the foreground region corresponding to the first video frame pixel data. The disclosed examples also include video encoder circuitry to generate a first foreground bounding region and a first background bounding region based on the segmentation data, determine a first virtual tile of the first video frame pixel data, the first virtual tile located in the first foreground bounding region, encode the first virtual tile into a video data bitstream without encoding the first background bounding region, and transmit the video data bitstream via a network.

First claim

What is claimed is:

1 . An apparatus comprising:
interface circuitry; machine-readable instructions; and at least one programmable circuit to be programmed based on the machine-readable instructions to:
generate a foreground bounding region and a background bounding region based on segmentation data associated with a video frame;
determine a virtual tile including first pixel data of the video frame, the virtual tile located in the foreground bounding region;
encode the first pixel data of the virtual tile into a video data bitstream without encoding second pixel data of the background bounding region into the video data bitstream; and
encode an identifier of a first virtual background of a plurality of virtual backgrounds in a message of the video data bitstream, the first virtual background to replace the background bounding region of the video frame.

2 . The apparatus of claim 1 , wherein one or more of the at least one programmable circuit is to generate an annotated region supplemental enhancement information message, the annotated region supplemental enhancement information message including at least one of size data, location data, or label data corresponding to the foreground bounding region and the background bounding region.

3 . The apparatus of claim 1 , wherein the foreground bounding region is a first foreground bounding region, the background bounding region is a first background bounding region, and one or more of the at least one programmable circuit is to:
create a second foreground bounding region and a second background bounding region based on an annotated region supplemental enhancement information message; determine a second virtual tile including third pixel data, the second virtual tile located in the second foreground bounding region; decode the second virtual tile; and generate video data corresponding to the second virtual tile and a second virtual background, the second virtual tile and the second virtual background to be displayed on a screen.

4 . The apparatus of claim 3 , wherein one or more of the at least one programmable circuit is to cause the decoded second virtual tile to be stored in a buffer pool in memory.

5 . The apparatus of claim 1 , wherein the background bounding region is a first background bounding region, and one or more of the at least one programmable circuit is to generate an annotated region supplemental enhancement information message, the annotated region supplemental enhancement information message including at least one of size data, location data, or label data corresponding to the foreground bounding region and a second background bounding region, the second background bounding region to overlap a portion of the foreground bounding region.

6 . At least one non-transitory computer-readable storage medium comprising instructions to cause at least one programmable circuit to at least:
generate a foreground bounding region and a background bounding region based on segmentation data associated with a video frame; determine a virtual tile including first pixel data of the video frame, the virtual tile in the foreground bounding region; encode the first pixel data of the virtual tile into a video data bitstream without encoding second pixel data of the background bounding region into the video data bitstream; and encode an identifier of a first virtual background of a plurality of virtual backgrounds in a message of the video data bitstream, the first virtual background to replace the background bounding region of the video frame.

7 . The computer-readable storage medium of claim 6 , wherein the instructions are to cause one or more of the at least one programmable circuit to generate an annotated region supplemental enhancement information message, the annotated region supplemental enhancement information message including at least one of size data, location data, or label data corresponding to the foreground bounding region and the background bounding region.

8 . The computer-readable storage medium of claim 6 , wherein the foreground bounding region is a first foreground bounding region, the background bounding region is a first background bounding region, and the instructions are to cause one or more of the at least one programmable circuit to:
create a second foreground bounding region and a second background bounding region based on an annotated region supplemental enhancement information message; determine a second virtual tile including third pixel data, the second virtual tile located in the second foreground bounding region; decode the second virtual tile; and generate a video signal corresponding to the second virtual tile and a second virtual background, the second virtual tile and the second virtual background to be displayed on a screen.

9 . The computer-readable storage medium of claim 8 , wherein the instructions are to cause one or more of the at least one programmable circuit to cause the decoded second virtual tile to be stored in a buffer pool in memory.

10 . The computer-readable storage medium of claim 6 , wherein the background bounding region is a first background bounding region, and the instructions are to cause one or more of the at least one programmable circuit to generate an annotated region supplemental enhancement information message, the annotated region supplemental enhancement information message including at least one of size data, location data, or label data corresponding to the foreground bounding region and a second background bounding region, the second background bounding region to overlap a portion of the foreground bounding region.

11 . An apparatus comprising:
means for generating segmentation data associated with a video frame; and means for encoding a virtual tile of the video frame, the means for encoding to:
generate a foreground bounding region and a background bounding region based on the segmentation data;
determine the virtual tile to include first pixel data of the video frame, the virtual tile located in the foreground bounding region; and
encode the first pixel data of the virtual tile into a video data bitstream without encoding second pixel data of the background bounding region into the video data bitstream; and
encode an identifier of a first virtual background of a plurality of virtual backgrounds in a message of the video data bitstream, the first virtual background to replace the background bounding region of the video frame.

12 . The apparatus of claim 11 , wherein the means for encoding is to generate an annotated region supplemental enhancement information message, the annotated region supplemental enhancement information message including at least one of size data, location data, or label data corresponding to the foreground bounding region and the background bounding region.

13 . The apparatus of claim 11 , including means for generating a video signal, wherein the foreground bounding region is a first foreground bounding region, the background bounding region is a first background bounding region, and:
the means for generating the segmentation data is to:
create a second foreground bounding region and a second background bounding region based on an annotated region supplemental enhancement information message;
determine a second virtual tile including third pixel data, the second virtual tile located in the second foreground bounding region; and
decode the second virtual tile; and
the means for generating the video signal is to generate the video signal based on the second virtual tile and a second virtual background, the second virtual tile and the second virtual background to be

Assignees

• Intel Corp

Inventors

• Guruva Reddiar Palanivel
• Boyce Jill
• Nair Praveen

Classifications

• H04N19/70

  characterised by syntax aspects related to video coding, e.g. related to compression standards · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• H04L65/403

  Arrangements for multi-party communication, e.g. for conferences (data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15) · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title

• H04N19/423

  characterised by memory arrangements (H04N19/433 takes precedence) · CPC title