Cache-based warp engine

Patent metadata

Field	Value
Publication number	US-12327296-B2
Application number	US-202117381873-A
Country	US
Kind code	B2
Filing date	Jul 21, 2021
Priority date	Jul 21, 2021
Publication date	Jun 10, 2025
Grant date	Jun 10, 2025

Abstract

The present invention relates to an image warping system capable of quickly performing image warping with low costs by using a cache memory, and a method thereof. The image warping system is provided to generate a transformed image by warping an input image with the help of a cache based WARP engine. The WARP engine accesses the input image and loads a portion of the image to the cache memory for speeding up the engine process. The WARP engine performs interpolation on the input image to generate an output image which is devoid of distortions. The output image obtained is then stored in the DDR of an electronic device.

First claim

The invention claimed is:

1. A method for warping an input image using a cache based WARP engine, wherein the method comprising:
accessing the input image from a DDR of a device;
loading a portion of the input image to a cache memory of the device;
pre-loading a lookup table stored in the cache memory, wherein the lookup table comprises a plurality of coordinates of the image, wherein the lookup table utilizes few kilo-bytes of the cache memory;
applying distortion correction and perspective correction to the input image;
mapping the input image by interpolating bi-cubically the plurality of coordinates to generate an interpolated image;
calculating an output image by raster scanning the interpolated image; and
storing the output image in the DDR of the device; and
generating an enhanced panoramic image without regard to image content by warping a panoramic digital image having skewed boundaries to fit a predetermined shape using a cache based WARP engine that is capable of supporting multiple image formats.

2. The method according to claim 1 , wherein the WARP engine decreasing the DDR access time and the DDR bandwidth requirement.

3. The method according to claim 1 , wherein the plurality of coordinates for the output image include 2-bytes for X-coordinates.

4. The method according to claim 1 , wherein the plurality of coordinates for the output image include 2-bytes for Y-coordinates.

5. The method according to claim 1 , wherein the lookup table is an 8×8 interval lookup table.

6. The method according to claim 1 , wherein the lookup table supports up to 4095×4095 image size.

7. The method according to claim 1 , wherein the device can be either of a tablet, a PC, a laptop, or a mobile phone.

8. The method according to claim 1 , wherein cache memory includes SRAM.

9. A system for warping an input image, wherein the system comprising:
an input device, wherein the input device comprising:
a DDR for storing the input image;
a cache memory with a plurality of cache lines, wherein the size of the cache memory is configurable and the plurality of cache lines access a portion of the image from the DDR;
a cache lookup table, wherein the cache look-up table comprises a plurality of coordinates of a portion of the input image; and

a cache based WARP engine, wherein the cache based WARP engine is capable of supporting different formats for the input image and is comprising:
 a pre-loader, wherein the pre-loader loads the cache lookup table with a set of coordinates based on the plurality of coordinates of the input image;
an image interpolator, wherein the image interpolator interpolates the input image bi-linearly or bi-cubically to generate an interpolated image;
a processor for calculating an output image by raster scanning the interpolated image; and
a memory for storing the output image on the DDR.

10. The system according to claim 9 , wherein the cache memory includes SRAM.

11. The system according to claim 9 , wherein the cache memory includes a cache tag.

12. The system according to claim 11 , further wherein the cache tag includes a horizontal tag and a vertical tag.

13. The system according to claim 11 , wherein the cache based follows the sequence of coordinates.

14. A non-transitory computer readable medium storing a computer program product comprising a computer useable medium having computer program logic for enabling at least one processor in a computer system for warping an image using a cache based WARP engine, said computer program logic comprising:
accessing the input image from a DDR of a device;
loading a portion of the input image to a cache memory of the device;
pre-loading a lookup table stored in the cache memory, wherein the lookup table comprises a plurality of coordinates of the image, wherein the lookup table utilizes few kilo-bytes of the cache memory;
applying distortion correction and perspective correction to the input image;
mapping the input image by interpolating bi-cubically the plurality of coordinates to generate an interpolated image;
calculating an output image by raster scanning the interpolated image; and
storing the output image in the DDR of the device; and
generating an enhanced panoramic image without regard to image content by warping a panoramic digital image having skewed boundaries to fit a predetermined shape using a cache based WARP engine that is capable of supporting multiple image formats.

Assignees

• Black Sesame International Holding Ltd
• Black Sesame Technologies Inc

Inventors

• Zhou Ying
• Wang Hui
• Li Jingyuan
• Huang Yanfang

Classifications

• G06T3/18

  Image warping, e.g. rearranging pixels individually · CPC title

• G06T3/4007

  based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) · CPC title

• G06F12/0877

  Cache access modes · CPC title

• G06T1/60 Primary

  Memory management · CPC title