Image processing method and image processing apparatus

Patent metadata

Field	Value
Publication number	US-12400382-B2
Application number	US-202218264739-A
Country	US
Kind code	B2
Filing date	Jan 4, 2022
Priority date	Feb 9, 2021
Publication date	Aug 26, 2025
Grant date	Aug 26, 2025

Abstract

The present disclosure provides an image processing method and an image processing apparatus. The image processing method including replacing an object in a scene image with a preset template image to generate a virtual image; dividing the scene image into a plurality of image blocks with a preset size; determining pixel residuals of the scene image according to an average pixel value of pixel points in an area where the object is located in the scene image and an average pixel value of pixel points falling into the area where the object is located in each of the plurality of image blocks; determining illumination offsets according to the pixel residuals; and updating pixel values of pixel points in the virtual image according to the illumination offsets to implement illumination transfer.

First claim

What is claimed is:

1. An image processing method, comprising:
replacing an object in a scene image with a preset template image to generate a virtual image;
dividing the scene image into a plurality of image blocks with a preset size;
determining pixel residuals of the scene image according to an average pixel value of pixel points in an area where the object is located in the scene image and an average pixel value of pixel points falling into the area where the object is located in each of the plurality of image blocks;
determining illumination offsets according to the pixel residuals; and
updating pixel values of pixel points in the virtual image according to the illumination offsets to implement illumination transfer.

2. The image processing method according to claim 1 , wherein the replacing the object in the scene image with the preset template image comprises:
mapping the template image into a pixel coordinate system of the scene image to generate a transformed image; and
replacing the object in the scene image with the transformed image to generate the virtual image.

3. The image processing method according to claim 2 , wherein the mapping the template image into the pixel coordinate system of the scene image comprises:
generating a minimum bounding rectangle of the object in the scene image;
mapping the minimum bounding rectangle from the pixel coordinate system to a preset plane coordinate system to obtain a first image;
transforming the first image into a square to obtain a second image;
determining a first transformation submatrix for mapping the template image to the second image;
determining a second transformation submatrix for mapping the first image to the minimum bounding rectangle;
obtaining a transformation matrix according to the first transformation submatrix and the second transformation submatrix; and
mapping the template image into the pixel coordinate system according to the transformation matrix.

4. The image processing method according to claim 3 , wherein
the transformation matrix is a product of the second transformation submatrix and the first transformation submatrix.

5. The image processing method according to claim 1 , wherein the determining the pixel residuals of the scene image comprises:
taking the average pixel value of the pixel points in the area where the object is located in the scene image as a reference average value;
taking the average pixel value of the pixel points falling into the area of the object in each of the plurality of image blocks as an average pixel value of the each of the plurality of image blocks; and
determining the pixel residuals of the scene image according to the average pixel value of the each of the plurality of image blocks and the reference average value.

6. The image processing method according to claim 5 , wherein the determining the illumination offsets according to the pixel residuals comprises:
performing interpolation processing on the pixel residuals to obtain the illumination offsets, wherein a size of a visual image corresponding to the illumination offsets is the same as a size of the virtual image.

7. The image processing method according to claim 1 , wherein the updating the pixel values of the pixel points in the virtual image according to the illumination offsets comprises:
calculating a first product of an offset D i,j corresponding to a pixel point P i,j at an i th row, a j th column of the virtual image in the illumination offsets and an illumination transfer adjusting parameter σ;
calculating a first sum of a pixel value R i,j 2 of a red channel of the pixel point P i,j and the first product, calculating a second sum of a pixel value G i,j 2 of a green channel of the pixel point P i,j and the first product, and calculating a third sum of a pixel value B i,j 2 of a blue channel of the pixel point P i,j and the first product; and
taking a maximum value of a second parameter and a minimum value of the first sum and a first parameter as a pixel value R i,j 3 of a red channel of a pixel point at an i th row, a j th column in a first transformed image, taking a maximum value of the second parameter and a minimum value of the second sum and the first parameter as a pixel value G i,j 3 of a green channel of the pixel point at the i th row, the j th column in the first transformed image, and taking a maximum value of the second parameter and a minimum value of the third sum and the first parameter as a pixel value B i,j 3 of a blue channel of the pixel point at the i th row, the j th column in the first transformed image.

8. The image processing method according to claim 1 , further comprising:
adjusting a saturation of a first transformed image according to the illumination offsets to generate a second transformed image after updating the pixel values of the pixel points in the virtual image according to the illumination offsets to generate the first transformed image; and
transforming pixel values of a hue channel, a saturation channel and a value channel in the hue, saturation and value (HSV) color space of the second transformed image into a red, green and blue (RGB) color space.

9. The image processing method according to claim 8 , wherein the adjusting the saturation of the first transformed image according to the illumination offsets comprises:
performing normalization processing on the illumination offsets;
calculating a second product of an offset D i,j n corresponding to a pixel point P i,j at an i th row, a j th column in the first transformed image in a normalized illumination offsets and a saturation adjusting parameter α;
calculating a difference between a third parameter and the second product; and
taking a maximum value of a fourth parameter and a third product of a pixel value S i,j 3 of the saturation channel of the pixel point P i,j in an i th row, a j th column in the first transformed image and the difference as a pixel value S i,j 3′ of a saturation channel of a pixel point at an i th row, a j th column in the second transformed image.

10. An image processing apparatus, comprising:
a memory; and
a processor coupled to the memory, which is configured to execute an image processing method comprising:
replacing an object in a scene image with a preset template image to generate a virtual image;
dividing the scene image into a plurality of image blocks with a preset size;
determining pixel residuals of the scene image according to an average pixel value of pixel points in an area where the object is located in the scene image and an average pixel value of pixel points falling into the area where the object is located in each of the plurality of image blocks;
determining illumination offsets according to the pixel residuals; and
updating pixel values of pixel points in the virtual image according to the illumination offsets to implement illumination transfer.

11. The image processing apparatus according to claim 10 , wherein the replacing the object in the scene image with the preset template image comprises:
mapping the template image into a pixel coordinate system of the scene image to generate a transformed image; and
replacing the object in the scene image with the transformed image to generate the virtual image.

12. The image processing apparatus according to claim 11 , wherein the mapping the template image into the pixel coordinate system of the scene image comprises:
generating a minimum bounding rectangle of the object in the scene image;
mapping the minimum bounding rectangle from the pixel coordinate system to a preset plane coordinate system to obtain a first image;

Assignees

• Beijing Wodong Tianjun Information Technology Co Ltd
• Beijing Jingdong Century Trading Co Ltd

Inventors

• Zhou Fangru
• Yang Mei
• An Shan

Classifications

• G06T2207/10024

  Color image · CPC title

• G06T15/00

  Three-dimensional [3D] image rendering · CPC title

• G06V10/60

  relating to illumination properties, e.g. using a reflectance or lighting model · CPC title

• G06V10/56

  relating to colour · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title