Method, electronic device, and computer program product for modeling an object utilizing a spatial distribution and differences between features of respective image pairs

Patent metadata

Field	Value
Publication number	US-12586306-B2
Application number	US-202318337537-A
Country	US
Kind code	B2
Filing date	Jun 20, 2023
Priority date	Jun 1, 2023
Publication date	Mar 24, 2026
Grant date	Mar 24, 2026

Abstract

A method for modeling an object in one embodiment includes: acquiring first position spatial information of a camera at a first position and second position spatial information of the camera at a second position, wherein the first position is different from the second position; generating a spatial distribution based on the first position spatial information and the second position spatial information, wherein the spatial distribution represents a probability distribution of space occupied by the object captured from a pose of the camera; generating a third image based on a first image captured at the first position, the spatial distribution, and the first position spatial information; generating a fourth image based on a second image captured at the second position, the spatial distribution, and the second position spatial information; and adjusting a model of the object based on the first image, the second image, the third image, and the fourth image.

First claim

What is claimed is:

1 . A method for modeling an object, comprising:
acquiring first position spatial information of a camera at a first position and second position spatial information of the camera at a second position, wherein the first position is different from the second position; generating a spatial distribution based on the first position spatial information and the second position spatial information, wherein the spatial distribution represents a probability distribution of space occupied by the object captured from a pose of the camera; generating a third image based on a first image captured at the first position, the spatial distribution, and the first position spatial information; generating a fourth image based on a second image captured at the second position, the spatial distribution, and the second position spatial information; and adjusting a model of the object based on the first image, the second image, the third image, and the fourth image; wherein adjusting the model based on the first image, the second image, the third image, and the fourth image comprises: calculating a first difference between features of the third image and features of the second image; calculating a second difference between features of the fourth image and features of the first image; and controlling adjustment of the model based on comparison of a function of at least the first difference and the second difference to at least one threshold.

2 . The method according to claim 1 , wherein generating the third image based on the first image captured at the first position, the spatial distribution, and the first position spatial information comprises:
generating a first model based on the first image and the spatial distribution; generating a second model by performing a grid transformation of the first model based on the pose of the camera in the first position spatial information; and generating the third image by performing a two-dimensional (2D) transformation of the second model.

3 . The method according to claim 2 , wherein generating the first model based on the first image and the spatial distribution comprises:
determining whether a probability value above a predetermined threshold of the spatial distribution exists at a position in the first image; and acquiring one or more of color information, texture information, and depth information at the position in response to the probability value above the predetermined threshold of the spatial distribution existing at the position in the first image.

4 . The method according to claim 2 , wherein generating the second model by performing the grid transformation of the first model based on the pose of the camera in the first position spatial information further comprises:
predicting features in the second model by applying a bicubic interpolation method on features in the first model.

5 . The method according to claim 1 , wherein generating the fourth image based on the second image captured at the second position, the spatial distribution, and the second position spatial information comprises:
generating a third model based on the second image and the spatial distribution; generating a fourth model by performing a grid transformation of the third model based on the pose of the camera in the second position spatial information; and generating the fourth image by performing a 2D transformation of the fourth model.

6 . The method according to claim 1 , wherein features of the third image correspond to features of the second image, and features of the fourth image correspond to features of the first image.

7 . The method according to claim 1 , wherein adjusting controlling adjustment of the model comprises:
continuing to adjust the model of the object in response to a sum of the first difference and the second difference being greater than a predetermined threshold; and stopping adjustment of the model of the object in response to the sum of the first difference and the second difference being smaller than the predetermined threshold.

8 . The method according to claim 1 , wherein adjusting the model based on the first image, the second image, the third image, and the fourth image further comprises:
adjusting the model of the object based on spatial density information, color information, texture information, and depth information in the first image, the second image, the third image, and the fourth image.

9 . The method according to claim 1 , wherein acquiring the first image based on the first position spatial information and acquiring the second image based on the second position spatial information comprises:
acquiring the first image and the second image by using the camera to shoot the same object at the first position and at the second position, respectively; and wherein the camera is jittered during use.

10 . An electronic device, comprising:
at least one processor; and memory coupled to the at least one processor and storing instructions, wherein the instructions, when executed by the at least one processor, cause the electronic device to perform actions comprising: acquiring first position spatial information of a camera at a first position and second position spatial information of the camera at a second position, wherein the first position is different from the second position; generating a spatial distribution based on the first position spatial information and the second position spatial information, wherein the spatial distribution represents a probability distribution of space occupied by an object captured from a pose of the camera; generating a third image based on a first image captured at the first position, the spatial distribution, and the first position spatial information; generating a fourth image based on a second image captured at the second position, the spatial distribution, and the second position spatial information; and adjusting a model of the object based on the first image, the second image, the third image, and the fourth image; wherein adjusting the model based on the first image, the second image, the third image, and the fourth image comprises: calculating a first difference between features of the third image and features of the second image; calculating a second difference between features of the fourth image and features of the first image; and controlling adjustment of the model based on comparison of a function of at least the first difference and the second difference to at least one threshold.

11 . The electronic device according to claim 10 , wherein generating the third image based on the first image captured at the first position, the spatial distribution, and the first position spatial information comprises:
generating a first model based on the first image and the spatial distribution; generating a second model by performing a grid transformation of the first model based on the pose of the camera in the first position spatial information; and generating the third image by performing a two-dimensional (2D) transformation of the second model.

12 . The electronic device according to claim 11 , wherein generating the first model based on the first image and the spatial distribution comprises:
determining whether a probability value above a predetermined threshold of the spatial distribution exists at a position in the first image; and acquiring one or more of color information, texture information, and depth information at the position in response to the probability value above the predetermined threshold of the spatial distribution existing at the position in the first image.

13 . The elect

Assignees

• Dell Products Lp

Inventors

• Liu Zhisong
• Wang Zijia
• Jia Zhen

Classifications

• G06T7/55

  from multiple images · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2207/10024

  Color image · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T7/70 Primary

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title