Methods and apparatus to warp images with decimation and interpolation

Patent metadata

Field	Value
Publication number	US-12354231-B2
Application number	US-202217589326-A
Country	US
Kind code	B2
Filing date	Jan 31, 2022
Priority date	Jan 31, 2022
Publication date	Jul 8, 2025
Grant date	Jul 8, 2025

Abstract

Methods, apparatus, systems, and articles of manufacture are disclosed to warp images for video processing. An example device includes horizontal warper circuitry configured to access an input image, decimate a first set of pixels in the input image by row, and interpolate a second set of pixels in the input image by row to produce a horizontally warped image. The example device further includes vertical warper circuitry configured to access the horizontally warped image, decimate a first set of pixels in the horizontally warped image by column, and interpolate a second set of pixels in the horizontally warped image by column to produce an output image.

First claim

What is claimed is:

1. A device comprising:
first warper circuitry configured to:
obtain an input image;
obtain a warp map;
in response to determining that a first group of pixels of the input image has a number of pixels greater than a first value:
decimate the first group of pixels responsive to the warp map, to produce a second group of pixels, the first group of pixels in a first direction; and
set a first warped image in a first direction to contain the second group of pixels; and

in response to determining that the first group of pixels of the input image has a number of pixels less than the first value:
set the second group of pixels to the first group of pixels; and
produce the first warped image based on interpolating the second group of pixels responsive to the warp map; and

second warper circuitry configured to:
obtain the first warped image;
obtain the warp map;
in response to determining that a third group of pixels of the first warped image has a number of pixels greater than a second value;
decimate the third group of pixels, responsive to the warp map, to produce a fourth group of pixels, the third group of pixels in a second direction; and
set an output image to contain the fourth group of pixels; and

in response to determining that the third group of pixels of the input image has a number of pixels less than the second value:
set the fourth group of pixels to the second third group of pixels; and
produce the output image based on interpolating the fourth group of pixels responsive to the warp map.

2. The device of claim 1 , further including:
configuration circuitry to partition the input image into a first plurality of rectangles; and
warp map circuitry configured to partition the output image into a second plurality of rectangles based on the first plurality of rectangles, the second plurality of rectangles based on a bilinear interpolation that includes a shift operation.

3. The device of claim 2 , wherein the warp map circuitry is to further partition the output image into a plurality of quadrilaterals such that:
each rectangle from the second plurality of rectangles corresponds to a quadrilateral; and
the second plurality of rectangles and the corresponding quadrilaterals are used to determine which pixels are decimated and interpolated.

4. The device of claim 1 , wherein to decimate and interpolate pixels, the first warper circuitry and the second warper circuitry are to implement separate polyphase filters for decimation and interpolation.

5. A method comprising:
decimating, by decimator circuitry, an input image in a direction, based on a warp map, to produce a decimated image;
partitioning a partial frame of memory into buffers based on the warp map;
storing pixel data from the decimated image in the buffers;
retrieving the decimated image from the buffer to produce a retrieved image; and
interpolating, by interpolator circuitry, the retrieved image in the direction, based on the warp map, to produce an output image.

6. The method of claim 5 , further including partitioning the partial frame of memory such that the buffers have a minimum length required to store the decimated image.

7. The method of claim 5 , further including interpolating the retrieved image in a First In First Out pattern.

8. The method of claim 5 , further including:
partitioning the input image into a first plurality of rectangles; and
partitioning the output image into a second plurality of rectangles based on the first plurality of rectangles, the second plurality of rectangles based on a bilinear interpolation that includes a shift operation.

9. The method of claim 8 , further including partitioning the output image into a plurality of quadrilaterals such that:
each rectangle from the second plurality of rectangles corresponds to a quadrilateral; and
the second plurality of rectangles and the corresponding quadrilaterals are used to determine which pixels are decimated and interpolated.

10. The method of claim 5 , further including implementing separate polyphase filters to decimate the input image and interpolate the retrieved image.

11. The method of claim 5 , wherein the decimating and the interpolating are mutually exclusive.

12. The method of claim 5 , wherein projecting the output image is by a projector, a Virtual Reality headset, or an Augmented Reality headset.

13. A method comprising:
obtaining, by warper circuitry, an input image;
obtaining, by the warper circuitry, a warp map;
warping, by the warper circuitry, the input image in a first direction to produce a first warped image, wherein the first direction is horizontal or vertical, and wherein warping the input image in a first direction comprises:
decimating pixels a first subset of rows or columns of the input image, responsive to the warp map, to produce a decimated image; and
interpolating a second subset of rows or columns of the decimated image, responsive to the warp map, to produce the first warped image, wherein the first subset of rows or columns and the second subset of rows or columns are mutually exclusive;

warping, by the warper circuitry, the first warped image in a second direction, responsive to the warp map, to produce an output image, wherein the second direction is horizontal or vertical, and wherein the second direction is different than the first direction; and
projecting the output image to produce a projected image.

14. The method of claim 13 , wherein the projected image is a foveated rendering of the input image and a first aspect ratio corresponding to the input image matches a second aspect ratio corresponding to the projected image.

15. The method of claim 13 , wherein the warper circuitry comprises an Application Specific Integrated Circuit (ASIC).

16. The method of claim 13 , further including:
partitioning the input image into a first plurality of rectangles; and
partitioning the output image into a second plurality of rectangles based on the first plurality of rectangles, the second plurality of rectangles based on a bilinear interpolation that includes a shift operation.

17. The method of claim 16 , further including partitioning the output image into a plurality of quadrilaterals such that:
each rectangle from the second plurality of rectangles corresponds to a quadrilateral; and
the second plurality of rectangles and the corresponding quadrilaterals are used to determine which pixels are horizontally warped and vertically warped.

18. The method of claim 13 , further including:
implementing decimation and interpolation operations to warp the input image and to warp the first warped image; and
implementing separate polyphase filters for the decimation and interpolation operations.

Assignees

• Texas Instruments Inc

Inventors

• Kempf Jeffrey
• Stroud Matthew C
• Fenske John

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G09G3/002

  to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2200/28

  involving image processing hardware · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title