Method, device, and computer program product for determining camera pose for an image

Patent metadata

Field	Value
Publication number	US-12561826-B2
Application number	US-202318319577-A
Country	US
Kind code	B2
Filing date	May 18, 2023
Priority date	Apr 13, 2023
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Embodiments of the present disclosure relate to a method, a device, and a computer program product for determining a camera pose for an image. The method includes: acquiring a query image for a target object. The method further includes: acquiring a set of images from a three-dimensional model for the target object. The method further includes: selecting a target image from the set of images based on similarities between images in the set of images and the query image. The method further includes: determining a target camera pose corresponding to the query image based on a camera pose for the target image. The method enables fast and accurate determination of a camera pose for a query image from a three-dimensional model for a target object, thus increasing the efficiency of acquiring the camera pose and improving the user experience.

First claim

What is claimed is:

1 . A method for determining a camera pose for an image, comprising:
acquiring a query image for a target object; acquiring a set of images from a three-dimensional model for the target object; selecting a target image from the set of images based on similarities between images in the set of images and the query image; and determining a target camera pose corresponding to the query image based on a camera pose for the target image; wherein acquiring the set of images, selecting the target image, and determining the target camera pose corresponding to the query image further comprise implementing a multi-stage process that comprises: in a first stage of the multi-stage process, sampling images from the three-dimensional model for a first number of views around a periphery of the three-dimensional model, and performing matching and sorting of the sampled images based on similarities of the respective sampled images to the query image to identify from the sampled images a first sampled image most similar to the query image; and in a second stage of the multi-stage process, sampling additional images from the three-dimensional model for a second number of views relative to a position of the first sampled image identified in the first stage, and repeating the matching and sorting for the additional sampled images based on similarities of the respective additional sampled images to the query image to identify from the additional sampled images a second sampled image most similar to the query image; wherein the three-dimensional model is adjusted based on the second sampled image.

2 . The method according to claim 1 , wherein acquiring a set of images comprises:
extracting the target object in the query image; determining the three-dimensional model for the target object based on the target object; and acquiring a set of images of the target object from the three-dimensional model, the set of images being images of the target object viewed from surroundings of the target object.

3 . The method according to claim 1 , wherein selecting a target image from the set of images comprises:
determining a similarity between each image of the set of images and the query image; and selecting, based on the similarities, an image having the highest similarity from the set of images as the target image.

4 . The method according to claim 3 , wherein determining a similarity between each image of the set of images and the query image comprises:
determining feature points in said each image and the query image; and determining the similarity based on the number of matching feature points in said each image and the query image.

5 . The method according to claim 1 , wherein the set of images is a first set of images, the target image is a first target image, and the camera pose is a first camera pose, and wherein determining a target camera pose corresponding to the query image comprises:
determining the first camera pose for the target image; acquiring, based on the first camera pose, a second set of images around a camera position corresponding to the first camera pose from the three-dimensional model; selecting a second target image from the second set of images based on similarities between images in the second set of images and the query image; and determining the target camera pose corresponding to the query image based on a second camera pose for the second target image.

6 . The method according to claim 1 , further comprising:
acquiring, based on the target camera pose, a reference image corresponding to the target camera pose from the three-dimensional model.

7 . The method according to claim 6 , further comprising:
determining, based on the query image and the reference image, a difference between the query image and the reference image; and presenting the difference on a display device.

8 . The method according to claim 1 , further comprising:
receiving a plurality of images for the target object; and generating the three-dimensional model based on the plurality of images.

9 . The method according to claim 8 , wherein generating the three-dimensional model comprises:
obtaining three-dimensional point cloud data for the target object based on the plurality of images; and generating the three-dimensional model based on the three-dimensional point cloud data.

10 . The method according to claim 9 , wherein obtaining three-dimensional point cloud data for the target object comprises:
generating three-dimensional point cloud data for a complete object corresponding to the plurality of images based on the plurality of images; and selecting the three-dimensional point cloud data for the target object from the three-dimensional point cloud data for the complete object.

11 . The method according to claim 9 , wherein obtaining three-dimensional point cloud data for the target object comprises:
selecting an image portion corresponding to the target object from each image of the plurality of images; and generating the three-dimensional point cloud data for the target object based on the selected image portion.

12 . The method according to claim 1 , wherein the three-dimensional model comprises a neural radiance field (NeRF) model.

13 . An electronic device, comprising:
at least one processor; and memory coupled to the at least one processor and having instructions stored therein, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising: acquiring a query image for a target object; acquiring a set of images from a three-dimensional model for the target object; selecting a target image from the set of images based on similarities between images in the set of images and the query image; and determining a target camera pose corresponding to the query image based on a camera pose for the target image; wherein acquiring the set of images, selecting the target image, and determining the target camera pose corresponding to the query image further comprise implementing a multi-stage process that comprises: in a first stage of the multi-stage process, sampling images from the three-dimensional model for a first number of views around a periphery of the three-dimensional model, and performing matching and sorting of the sampled images based on similarities of the respective sampled images to the query image to identify from the sampled images a first sampled image most similar to the query image; and in a second stage of the multi-stage process, sampling additional images from the three-dimensional model for a second number of views relative to a position of the first sampled image identified in the first stage, and repeating the matching and sorting for the additional sampled images based on similarities of the respective additional sampled images to the query image to identify from the additional sampled images a second sampled image most similar to the query image; wherein the three-dimensional model is adjusted based on the second sampled image.

14 . The electronic device according to claim 13 , wherein acquiring a set of images comprises:
extracting the target object in the query image; determining the three-dimensional model for the target object based on the target object; and acquiring a set of images of the target object from the three-dimensional model, the set of images being images of the target object viewed from surroundings of the target object.

15 . The electronic device according to claim 13 , wherein selecting a target image from the set of images comp

Assignees

• Dell Products Lp

Inventors

• Liu Zhisong
• Wang Zijia
• Jia Zhen

Classifications

• G06V10/44

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06F16/583

  using metadata automatically derived from the content · CPC title

• G06V10/761 Primary

  Proximity, similarity or dissimilarity measures · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T17/00

  Three-dimensional [3D] modelling for computer graphics · CPC title