Image processing method and apparatus, device, storage medium, and computer program product

Patent metadata

Field	Value
Publication number	US-2023342973-A1
Application number	US-202318207018-A
Country	US
Kind code	A1
Filing date	Jun 7, 2023
Priority date	Nov 24, 2021
Publication date	Oct 26, 2023
Grant date	—

Abstract

Embodiments of this application provide an image processing method performed by a computer device. The method includes: obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern; determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively; determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; and fusing a target object image and a virtual background image in the actual image based on intrinsic information and the pose information to obtain a target image.

First claim

What is claimed is:

1 . An image processing method, performed by a computer device, the method comprising:
obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern; determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively; determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; and fusing the target object image and a virtual background image based on the intrinsic information and the target pose information to obtain a target image.

2 . The method according to claim 1 , further comprising:
in response to the actual image not comprising a position reference pattern, obtaining historical pose information of the camera in the world coordinate system when the camera photographs a previous actual image as the target pose information.

3 . The method according to claim 1 , wherein the determining first position information of the position reference pattern in an image coordinate system comprises:
obtaining at least one feature point contained in the position reference pattern and a first feature point coordinate of each feature point in the image coordinate system; and taking the obtained at least one feature point and the first feature point coordinate as the first position information of the position reference pattern.

4 . The method according to claim 1 , wherein the determining second position information of the position reference pattern in a world coordinate system comprises:
obtaining a pattern number embedded in the position reference pattern; querying a feature point number of each feature point and a second feature point coordinate of each feature point in the world coordinate system based on the pattern number; and taking the queried feature point number of each feature point and the second feature point coordinate of each feature point as the second position information of the position reference pattern.

5 . The method according to claim 1 , wherein the actual image further comprises a background image, and the background image comprises a green screen region and a non-green screen region; and
before the fusing the target object image and a virtual background image based on the intrinsic information and the target pose information to obtain a target image, the method further comprises:
obtaining third position information of a boundary line between the green screen region and the non-green screen region in the world coordinate system;
determining fourth position information of the boundary line in the image coordinate system based on the third position information, the intrinsic information, and the target pose information;
removing an image of the green screen region in the actual image to obtain a pre-processed image; and
removing, based on the fourth position information, an image of the non-green screen region located outside the boundary line in the pre-processed image to obtain the target object image.

6 . The method according to claim 1 , wherein the fusing the target object image and a virtual background image based on the intrinsic information and the target pose information to obtain a target image comprises:
determining a field of view of the camera based on the intrinsic information; determining attribute information of a virtual photographing device in a virtual world through a rendering engine based on the intrinsic information, the target pose information, and the field of view; determining target position information of a target imaging plane of the target object image in the virtual world based on the attribute information of the virtual photographing device and a reference position point in the target object image; and fusing the target object image and the virtual background image based on the target position information of the target imaging plane to obtain the target image.

7 . The method according to claim 6 , wherein the fusing the target object image and the virtual background image based on the target position information of the target imaging plane to obtain the target image comprises:
determining target size information of the target imaging plane based on the attribute information of the virtual photographing device and the target position information of the target imaging plane; and fusing the target object image and the virtual background image based on the target position information and the target size information to obtain the target image.

8 . A computer device, comprising a memory and a processor,
the memory being configured to store computer programs executed on the processor; and the processor being configured to, when executing the computer programs, cause the computer device to perform an image processing method including:
obtaining an actual image photographed by a camera, the actual image comprising a target object image and a position reference pattern;
determining first position information of the position reference pattern in an image coordinate system and second position information of the position reference pattern in a world coordinate system, respectively;
determining target pose information of the camera in the world coordinate system based on the first position information, the second position information, and intrinsic information of the camera; and
fusing the target object image and a virtual background image based on the intrinsic information and the target pose information to obtain a target image.

9 . The computer device according to claim 8 , wherein the method further comprises:
in response to the actual image not comprising a position reference pattern, obtaining historical pose information of the camera in the world coordinate system when the camera photographs a previous actual image as the target pose information.

10 . The computer device according to claim 8 , wherein the determining first position information of the position reference pattern in an image coordinate system comprises:
obtaining at least one feature point contained in the position reference pattern and a first feature point coordinate of each feature point in the image coordinate system; and taking the obtained at least one feature point and the first feature point coordinate as the first position information of the position reference pattern.

11 . The computer device according to claim 8 , wherein the determining second position information of the position reference pattern in a world coordinate system comprises:
obtaining a pattern number embedded in the position reference pattern; querying a feature point number of each feature point and a second feature point coordinate of each feature point in the world coordinate system based on the pattern number; and taking the queried feature point number of each feature point and the second feature point coordinate of each feature point as the second position information of the position reference pattern.

12 . The computer device according to claim 8 , wherein the actual image further comprises a background image, and the background image comprises a green screen region and a non-green screen region; and
before the fusing the target object image and a virtual background image based on the intrinsic information and the target pose information to obtain a target image, the method further comprises:
obtaining third position information of a boundary line between the

Assignees

• Tencent Tech Shenzhen Co Ltd

Inventors

• Chen Fasheng
• Lin Zhiyang
• Sun Lei

Classifications

• G06T7/73 Primary

  using feature-based methods · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/13

  Edge detection · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G06T2207/20221

  Image fusion; Image merging · CPC title