Multi-block memory reads for image de-warping

Patent metadata

Field	Value
Publication number	US-10037598-B2
Application number	US-201615363870-A
Country	US
Kind code	B2
Filing date	Nov 29, 2016
Priority date	Nov 29, 2016
Publication date	Jul 31, 2018
Grant date	Jul 31, 2018

Abstract

Techniques related to multi-block memory reads for image de-warping are discussed. Such techniques may include copying pixel data corresponding to overlapping regions of memory between adjacent image modification regions from a local buffer and retrieving pixel data corresponding to non-overlapping regions from external memory.

First claim

What is claimed is:

1. A machine based method for attaining pixel data for image de-warping comprising:
storing pixel data corresponding to at least a portion of an input image in a memory;
determining, for a current image modification region of the input image, a corresponding first region of the memory storing first pixel data corresponding to the current image modification region;
comparing the first region of the memory to a second region of the memory corresponding to second pixel data of a previous image modification region of the input image to generate an overlapping region of the memory that is in both the first region and the second region;
copying third pixel data corresponding to the overlapping region of the memory from a first internal buffer to a second internal buffer, wherein the first and second internal buffers are separate from the memory;
retrieving fourth pixel data corresponding to a non-overlapping region of the first region of the memory from the memory and storing the fourth pixel data to the second internal buffer; and
performing distortion correction based on accessing at least on the third and fourth pixel data in the second internal buffer to generate a portion of a distortion corrected output image.

2. The method of claim 1 , wherein retrieving the fourth pixel data comprises retrieving the fourth pixel data based on a plurality of ordered memory access commands, the method further comprising:
determining a spatial direction corresponding to the current image modification region with respect to a coordinate system of the input image; and
ordering the plurality of ordered memory access commands in a memory direction corresponding to the spatial direction, wherein the memory direction is with respect to a coordinate system of the memory.

3. The method of claim 2 , wherein the coordinate system of the input image is aligned with a capture or presentation orientation of the input image and the coordinate system of the memory provides for storing pixel data of the input image in a layout matching the coordinate system of the input image.

4. The method of claim 1 , wherein the current image modification region corresponds to a plurality of points of a sparse distortion correction grid and a filter support region at least partially surrounding the plurality of points.

5. The method of claim 4 , wherein the plurality of points define a tetragon having four corner grid points and the filter support region comprises a region around the tetragon to support a 2-dimensional pixel filter.

6. The method of claim 1 , further comprising:
determining at least a first and a second region of interest of the input image, wherein each of the first and second regions of interest comprises an image region and a filter support region at least partially surrounding the image region; and
grouping at least the first and second of regions of interest to generate the current image modification region.

7. The method of claim 1 , further comprising:
determining, for a portion of a sparse distortion grid corresponding to at least the current image modification region, a local distortion measurement;
determining a region grouping factor for the portion of the sparse distortion grid based on the local distortion measurement; and
grouping a number of regions of interest equal to the region grouping factor to generate the current image modification region.

8. The method of claim 7 , wherein determining the region grouping factor comprises:
comparing the local distortion measurement to at least one threshold; and
providing a first region grouping factor when the local distortion measurement is greater than the threshold and a second region grouping factor greater than the first region grouping factor when the local distortion measurement is less than the threshold.

9. The method of claim 1 , wherein the third pixel data is previously fetched from the memory with respect to memory access commands corresponding to retrieving the fourth pixel data.

10. The method of claim 1 , wherein the memory comprises dynamic random access memory providing pixel data storage by cache line in a raster order.

11. A system for image de-warping comprising:
memory to store pixel data corresponding to at least a portion of an input image; and
an image signal processor coupled to the memory, the image signal processor comprising a first internal buffer, a second internal buffer, a microcontroller, and a distortion correction processor, the microcontroller to determine, for a current image modification region of the input image, a corresponding first region of the memory storing first pixel data corresponding to the current image modification region, to compare the first region of the memory to a second region of the memory corresponding to second pixel data of a previous image modification region of the input image to generate an overlapping region of the memory that is in both the first region and the second region, to generate a command to copy third pixel data corresponding to the overlapping region from the first internal buffer to the second internal buffer, and to generate one or more commands to retrieve fourth pixel data corresponding to a non-overlapping region of the first region from the memory to the second internal buffer, the second internal buffer to store the third and the fourth pixel data responsive to the commands, and the distortion correction processor to perform distortion correction based at least on the third and fourth pixel data in the second internal buffer to generate a portion of a distortion corrected output image.

12. The system of claim 11 , wherein the one or more commands to retrieve the fourth pixel data comprise ordered memory access commands, wherein the microcontroller is to determine a spatial direction corresponding to the current region of interest with respect to a coordinate system of the input image and to order the plurality of ordered memory access commands in a memory direction corresponding to the spatial direction, wherein the memory direction is with respect to a coordinate system of the memory.

13. The system of claim 12 , wherein the coordinate system of the input image is aligned with a capture or presentation orientation of the input image and the coordinate system of the memory provides for storing pixel data of the input image in a layout matching the coordinate system of the input image.

14. The system of claim 11 , wherein the current image modification region corresponds to a plurality of points of a sparse distortion correction grid and a filter support region at least partially surrounding the plurality of points.

15. The system of claim 11 , wherein the microcontroller is further to determine at least a first and a second region of interest of the input image, wherein each of the first and second regions of interest comprises an image region and a filter support region at least partially surrounding the image region, and to group at least the first and second of regions of interest to generate the current image modification region.

16. The system of claim 11 , wherein the microcontroller is further to determine, for a portion of a sparse distortion grid corresponding to at least the current image modification region, a local distortion measurement, to determine a region grouping factor for the portion of the sparse distortion grid based on the local distortion measurement, and to group a number of regions of interest equal to the region grouping factor to generate the current region of interest of the input image.

17. The system of clai

Assignees

• Intel Corp

Inventors

• Gur Amit
• Kol Rakefet
• Van Dalen Edwin
• Klein Tamir

Classifications

• G06T5/006 Primary

  Physics · mapped topic

• G06K9/4604

  Physics · mapped topic

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T3/0093

  Physics · mapped topic

• G06T1/60

  Memory management · CPC title