Method and apparatus for enhanced anti-aliasing filtering on a GPU

Patent metadata

Field	Value
Publication number	US-11508043-B2
Application number	US-202016904952-A
Country	US
Kind code	B2
Filing date	Jun 18, 2020
Priority date	Jun 18, 2020
Publication date	Nov 22, 2022
Grant date	Nov 22, 2022

Abstract

A method for reducing abasing artefacts in an output image may include obtaining a plurality of input images captured by a plurality of cameras, each camera having a different field of view of an environment surrounding a vehicle, wherein the plurality of input images are mapped to the output image to represent the environment, from a predefined virtual point of view. The method may further include for each pixel position in the output image, obtaining a first pixel density value corresponding to a first output pixel position in the output image; and upon determining that the first pixel density value is higher than a threshold, calculating a first output brightness value corresponding to the first output pixel position based at least on a plurality of brightness values corresponding to a plurality of neighboring pixels of a corresponding position in a first input image of the plurality of input images.

First claim

What is claimed is:

1. A method for reducing aliasing artefacts in an output image, comprising:
obtaining a plurality of input images captured by a plurality of cameras, each camera having a different field of view of an environment surrounding a vehicle, wherein the plurality of input images are mapped to the output image such that the output image represents the environment from a predefined virtual point of view; and
for each output pixel position in the output image: obtaining a first pixel density value corresponding to a first output pixel position in the output image; and
upon determining that the first pixel density value is higher than a threshold, calculating a first output brightness value corresponding to the first output pixel position in the output image based at least on a plurality of brightness values corresponding to a plurality of neighboring pixels of a corresponding position in a first input image of the plurality of input images,
wherein the first pixel density value determines an amount of sub-sampling or up-sampling of image content when mapping the plurality of input images to the first output pixel position in the output image through a target viewport, and
determining that a strong anti aliasing mechanism is needed when the first pixel density value is greater than a predetermined pixel density value, wherein the strong anti-aliasing mechanism is performed by shifting positions of the plurality of neighboring pixels of the corresponding position in the first input image of the plurality of input images toward respective group centers and by adapting weights for the plurality of neighboring pixels correspondingly.

2. The method of claim 1 , further comprising:
upon determining that the first pixel density value is equal to or lower than a threshold, calculating the first output brightness value corresponding to the first output pixel position in the output image based on a brightness value of a pixel in the first input image of the plurality of input images corresponding to the first output pixel position in the output image.

3. The method of claim 1 , wherein obtaining the first pixel density value corresponding to the first output pixel position in the output image comprises:
selecting a second output pixel directly to the left of the first output pixel position in the output image, and selecting a third output pixel directly to the right of the first output pixel position in the output image;
determining a second input pixel position and a third input pixel position in the first input image, corresponding to the second and third output pixel positions in the output image, respectively;
calculating a first distance value between the second input pixel position and the third input pixel position;
selecting a fourth output pixel directly above the first output pixel position in the output image, and selecting a fifth output pixel directly to the bottom of the first output pixel position in the output image;
determining a fourth input pixel position and a fifth input pixel position in the first input image, corresponding to the fourth and fifth output pixel positions in the output image, respectively;
calculating a second distance value between the fourth input pixel position and the fifth input pixel position; and
calculating the first pixel density value corresponding to the first output pixel position in the output image as a function of a maximum of the first distance value and the second distance value.

4. The method of claim 1 , wherein obtaining the first pixel density value comprises:
receiving the predetermined pixel density value corresponding to the first output pixel position in the output image.

5. The method of claim 1 , wherein the first pixel density value represents a measure of number of pixels in an input image that are mapped between the first output pixel in the output image in the output image and a second pixel directly adjacent to the first output pixel in the output image.

6. The method of claim 1 , wherein determining the first output pixel brightness value corresponding to the first output pixel position in the output image comprises:
calculating the first output brightness value corresponding to the first output pixel position in the output image as a weighted average of the plurality of brightness values corresponding to the plurality of neighboring pixel values in the first input image.

7. The method of claim 6 , wherein a weight corresponding to each brightness value is calculated based on a distance between a position in the first input image corresponding to the first output pixel position in the output image and each of four pixels directly adjacent to the position in the first input image.

8. The method of claim 6 , wherein the plurality of brightness values comprise prefiltered brightness values corresponding to each of the plurality of the neighboring pixel values in the first input image, wherein the prefiltered brightness values are calculated by a graphics processing unit (GPU) by means of bilinear interpolation.

9. An apparatus for reducing aliasing artefacts in an output image, comprising at least one processor coupled to a memory, the at least one processor is configured to:
obtain a plurality of input images captured by a plurality of cameras, each camera having a different field of view of an environment surrounding a vehicle, wherein the plurality of input images are mapped to the output image such that the output image represents the environment from a predefined virtual point of view; and
for each output pixel position in the output image: obtain a first pixel density value corresponding to a first output pixel position in the output image;
and upon determining that the first pixel density value is higher than a threshold, calculate a first output brightness value corresponding to the first output pixel position in the output image based at least on a plurality of brightness values corresponding to a plurality of neighboring pixels of a corresponding position in a first input image of the plurality of input images,
wherein the first pixel density value determines an amount of sub-sampling or up-sampling of image content when mapping the plurality of input images to the first output pixel position in the output image through a target viewport, and
determining that a strong anti-aliasing mechanism is needed when the first pixel density value is greater than a predetermined pixel density value, wherein the strong anti-aliasing mechanism is performed by shifting positions of the plurality of neighboring pixels of the corresponding position in the first input image of the plurality of input images toward respective group centers and by adapting weights for the plurality of neighboring pixels correspondingly.

10. The apparatus of claim 9 , wherein the at least one processor is further configured to:
upon determining that the first pixel density value is equal to or lower than a threshold, calculate the first output brightness value corresponding to the first output pixel position in the output image based on a brightness value of a pixel in the first input image of the plurality of input images corresponding to the first output pixel position in the output image.

11. The apparatus of claim 9 , wherein the at least one processor configured to obtain the first pixel density value corresponding to the first output pixel position in the output image is configured to:
select a second output pixel directly to the left of the first output pixel position in the output image, and selecting a third output pixel directly to the right of the first output pixel position in the output image;
determine a second input pixel posi

Assignees

• Connaught Electronics Ltd

Inventors

• Griffin Mark

Classifications

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/71

  Circuitry for evaluating the brightness variation · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title