Image processing method and apparatus with adaptive sampling

Patent metadata

Field	Value
Publication number	US-9865078-B2
Application number	US-201514939263-A
Country	US
Kind code	B2
Filing date	Nov 12, 2015
Priority date	Apr 23, 2015
Publication date	Jan 9, 2018
Grant date	Jan 9, 2018

Abstract

According to a method of processing an image, respective sampling rates of each tile included in a second frame is determined based on a corresponding rendering result of a first frame, and a pixel rendering process to implement rendering of pixels included in the second frame is selected by using geometric information of reference pixels of the second frame. The reference pixels of the second frame may be selected based on the respectively determined sampling rates.

First claim

What is claimed is:

1. An image processing method, the method comprising:
rendering, using at least one processor, a tile of a first frame of an image using a first sampling rate;
determining, using the at least one processor, a second sampling rate for a tile of a second frame of the image corresponding to the tile of the first frame based on a rendering result of the first frame;
dividing, using the at least one processor, the tile of the second frame into a number of pixel blocks, each of the pixel blocks including a plurality of pixels, based on the determined second sampling rate;
selecting, using the at least one processor, reference pixels from the pixel blocks based on the determined second sampling rate;
obtaining, using the at least one processor, geometric information of the reference pixels by performing ray tracing on the reference pixels;
obtaining, using the at least one processor, geometric information of an intermediate pixel of the tile by selecting, based on the obtained geometric information of the reference pixels, a pixel rendering process to render the intermediate pixel; and
performing, using the at least one processor, the selected pixel rendering process for the intermediate pixel.

2. The method of claim 1 , wherein the selecting of the pixel rendering process includes:
selecting between one of ray tracing the intermediate pixel to obtain the geometric information of the intermediate pixel, and interpolating the intermediate pixel based on two or more of the reference pixels to obtain the geometric information of the intermediate pixel.

3. The method of claim 1 , wherein the second sampling rate of the tile of the second frame is selectively, based on a determined similarity of the tile of the first frame that corresponds to the tile of the second frame, determined to be one of lower than the first sampling rate used to render the corresponding tile of the first frame, higher than the first sampling rate used to render the corresponding tile of the first frame, and same as the first sampling rate used to render the corresponding tile of the first frame.

4. The method of claim 1 , wherein, in the determining of the second sampling rate for the tile of the second frame,
the second sampling rate for the tile of the second frame is determined to be lower than the first sampling rate used to render the corresponding tile of the first frame when a determined similarity of the corresponding tile of the first frame is determined to meet a first threshold; and
the second sampling rate for the tile of the second frame is determined to be higher than the first sampling rate used to render the corresponding tile of the first frame when the determined similarity of the corresponding tile of the first frame is determined to not meet a second threshold.

5. The method of claim 4 , wherein the determined similarity of the corresponding tile of the first frame is a previously determined similarity between selected reference pixels of the corresponding tile of the first frame calculated based on obtained geometric information of the selected reference pixels of the corresponding tile of the first frame and intermediate pixels of the first frame adjacent to the selected reference pixels of the corresponding tile of the first frame.

6. The method of claim 1 , further comprising:
obtaining, using the at least one processor, a previously determined similarity and a previously determined importance of the tile of the first frame,
wherein the determining of the second sampling rate for the tile of the second frame comprises determining the second sampling rate for the tile of the second frame based on the obtained similarity and the obtained importance.

7. The method of claim 6 , wherein the determining of the second sampling rate for the tile of the second frame is based on a determination of whether the obtained similarity is determined to meet a threshold, with the threshold being greater when the obtained importance of the first tile meets an importance threshold compared to when the obtained importance of the first tile is determined to not meet the importance threshold.

8. The method of claim 1 , further comprising:
obtaining, using the at least one processor, a previously determined similarity and previously determined importance of the tile of the first frame that is determined to correspond to the tile of the second frame,
wherein the selecting of the pixel rendering process comprises selecting the pixel rendering process based on a determined similarity between the reference pixels, the determination of the similarity between the reference pixels being dependent on the obtained importance.

9. The method of claim 8 , wherein the selecting of the pixel rendering process further comprises:
determining whether the obtained importance meets an importance threshold; and
based on a result of the determining of whether the obtained importance meets the importance threshold, selecting between using one of a greater, lesser, or same similarity threshold between the reference pixels to select a first pixel rendering process, instead of selecting a second pixel rendering process, to render the intermediate pixel compared to a similarity threshold between reference pixels of the tile of the first frame that was used to control which of the first pixel rendering process and the second pixel rendering process was selected to render an intermediate pixel of the tile of the first frame.

10. The method of claim 1 , further comprising:
obtaining, using the at least one processor, a previously determined similarity and previously determined importance of the tile of the first frame that is determined to correspond to the tile of the second frame,
wherein the determining of the second sampling rate of the second tile comprises determining the second sampling rate of the second tile based on the obtained similarity and the obtained importance, and
the selecting of the pixel rendering process comprises selecting the pixel rendering process based on the obtained importance and a determined similarity between the reference pixels of the second tile.

11. The method of claim 1 , further comprising:
dividing, using the at least one processor, each of plural tiles included in the first frame into pixel blocks, respectively including a plurality of pixels, and selecting a respective reference pixel of the first frame from each of the pixel blocks;
calculating, using the at least one processor, respective similarities between the reference pixels of the first frame and/or intermediate pixels of the first frame adjacent to the reference pixels of the first frame based on determined geometric information of the reference pixels of the first frame; and
selecting, using the at least one processor, respective pixel rendering processes to render the intermediate pixels of the first frame based on the calculated respective similarities,
wherein the determining of the second sampling rate for the second tile comprises determining respective second sampling rates for each tile included in the second frame based on the calculated respective similarities.

12. A non-transitory computer-readable recording medium comprising computer readable code to control at least one processing device to implement the method of claim 1 .

13. An image processing apparatus, the apparatus comprising:
a memory having computer readable instructions stored thereon; and
at least one processor configured to execute the computer readable instructions to,
render a tile of a first frame of an image using a first sampling rate;
determine a second sampling rate for a of tile of a second f

Assignees

• Samsung Electronics Co Ltd

Inventors

• Lee Wonjong
• Shin Youngsam
• Hwang Seokjoong

Classifications

• G06T15/06 Primary

  Ray-tracing · CPC title

• G06T15/005

  General purpose rendering architectures · CPC title