Systems and methods for encoding image data

Patent metadata

Field	Value
Publication number	US-11330296-B2
Application number	US-202017020750-A
Country	US
Kind code	B2
Filing date	Sep 14, 2020
Priority date	Sep 14, 2020
Publication date	May 10, 2022
Grant date	May 10, 2022

Abstract

Systems and methods for improving operational efficiency of a video encoding system used to encode image data are provided. In embodiments, a video encoding system includes image processing circuitry configured to receive source image data and derive full-resolution image data and low-resolution image data from the source image data. The video encoding system also includes a low resolution pipeline configured to receive the low-resolution image data and determine one or more low resolution inter prediction modes based on the low-resolution image data. Furthermore, the video encoding system includes a main pipeline configured to encode the full-resolution image data based on the one or more low resolution inter prediction modes.

First claim

What is claimed is:

1. A video encoding system comprising:
image processing circuitry configured to:
receive source image data; and
derive full-resolution image data and low-resolution image data from the source image data;

memory configured to store the low-resolution image data;
a low resolution pipeline separate from the image processing circuitry and the memory, wherein the low resolution pipeline is configured to receive the low-resolution image data from the memory without further downscaling and, after receiving the low-resolution image data from the memory, determine one or more low resolution inter prediction modes based on the low-resolution image data; and
a main pipeline separate from the image processing circuitry and configured to encode the full-resolution image data based on the one or more low resolution inter prediction modes.

2. The video encoding system of claim 1 , wherein:
the memory is configured to store the full-resolution image data; and
the main pipeline is configured to encode the full-resolution image data after receiving the full-resolution image data from the memory.

3. The video encoding system of claim 1 , wherein the image processing circuitry is configured to derive the full-resolution image data and the low-resolution image data by downscaling the source image data.

4. The video encoding system of claim 1 , wherein the image processing circuitry is configured to derive the full-resolution image data by upscaling the source image data.

5. The video encoding system of claim 1 , wherein the full-resolution image data comprises one or more frames of image data having 4K resolution or higher.

6. The video encoding system of claim 1 , wherein:
the source image data comprises data for a first number of pixels;
the full-resolution image data comprises data for a second number of pixels;
the low-resolution image data comprises data for a third number of pixels; and
a sum of the second number of pixels and the third number of pixels is less than the first number of pixels.

7. The video encoding system of claim 1 , wherein the low-resolution image data comprises a first resolution that is one sixteenth of a second resolution of the full-resolution image data.

8. The video encoding system of claim 1 , wherein the video encoding system is included within a portable phone, a media player, a personal data organizer, a handheld game platform, a tablet device, a computer, or any combination thereof.

9. The video encoding system of claim 1 , wherein:
the memory is configured to store the full-resolution image data; and
the main pipeline is configured to encode the full-resolution image data after receiving the full-resolution image data from the memory.

10. A method, comprising:
receiving, via image processing circuitry, source image data having a first resolution;
deriving, via the image processing circuitry, full-resolution image data having a second resolution and low-resolution image data having a third resolution from the source image data;
storing the full-resolution image data and the low-resolution image data in memory;
retrieving, via a low resolution motion estimation block of an encoding system that is separate from the image processing circuitry and the memory, the low-resolution image data from the memory without performing additional downscaling; and
after retrieving the low-resolution image data from the memory, encoding, via the encoding system, the full-resolution image data based on one or more determinations made utilizing the low-resolution image data.

11. The method of claim 10 , wherein the second resolution is lower than the first resolution.

12. The method of claim 11 , wherein the third resolution is lower than the second resolution.

13. The method of claim 10 , comprising making, via the low resolution motion estimation block, the one or more determinations by determining one or more lower resolution inter prediction modes based on the low-resolution image data.

14. The method of claim 10 , wherein receiving the source image data comprises receiving the source image data from an image sensor communicatively coupled to the image processing circuitry.

15. The method of claim 10 , wherein deriving the full-resolution image data and the low-resolution image data comprises downscaling the source image data.

16. An electronic device, comprising:
image processing circuitry configured to:
receive source image data; and
derive full-resolution image data and low-resolution image data from the source image data;

memory communicatively coupled to the image processing circuitry and configured to store the low-resolution image data and the full-resolution image data;
a low resolution pipeline separate from the image processing circuitry and the memory, wherein the low resolution pipeline is configured to receive the low-resolution image data from the memory without performing additional downscaling on the low-resolution image data and, after receiving the low-resolution image data from the memory determine one or more low resolution inter prediction modes based on the low-resolution image data; and
a main pipeline separate from the image processing circuitry and the memory, wherein the main pipeline is configured to receive the full-resolution image data from the memory and encode the full-resolution image data received from the memory based on the one or more low resolution inter prediction modes.

17. The electronic device of claim 16 , comprising an image sensor configured to generate the source image data.

18. The electronic device of claim 17 , wherein the image sensor comprises a camera.

19. The electronic device of claim 16 , wherein the image processing circuitry is configured to derive the low-resolution image data from the full-resolution image data.

20. The electronic device of claim 16 , wherein the electronic device comprises a portable phone, a media player, a personal data organizer, a handheld game platform, a tablet device, a computer, or any combination thereof.

Assignees

• Apple Inc

Inventors

• Côté Guy
• Leontaris Athanasios
• Wang Muge

Classifications

• H04N19/182

  the unit being a pixel · CPC title

• H04N19/136

  Incoming video signal characteristics or properties · CPC title

• H04N19/103 Primary

  Selection of coding mode or of prediction mode · CPC title

• H04N19/59 Primary

  involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution · CPC title

• H04N19/517

  by encoding · CPC title