Reconstruction of 3D model with immersive experience

Patent metadata

Field	Value
Publication number	US-12260509-B2
Application number	US-202318216583-A
Country	US
Kind code	B2
Filing date	Jun 29, 2023
Priority date	Jun 12, 2018
Publication date	Mar 25, 2025
Grant date	Mar 25, 2025

Abstract

A system receives image data associated with an item, where the image data comprising a view of the item from two or more angles; determines physical attributes of the item; generates a base model of the item; samples the base model to generate one or more sampled models, each of the one or more sampled models comprising a subset of the geometric data, the subset of the geometric data determined based on one or more device characteristics of one or more user devices that interface with the system; receives device characteristics of a user device associated with a request from the user device for the item; selects, based on the received device characteristics, a sampled model of the item; and transmits a data object comprising the selected sampled model to the user device to cause the user device to generate a three-dimensional rendering of the item.

First claim

What is claimed is:

1. A method performed at a server, the method comprising:
extracting points from each of a plurality of images depicting different two-dimensional orientations of an item, to generate geometric data comprising geometric primitives and visual elements of the item;
generating, using the geometric data, a base model of the item;
receiving, from a user device, a request to view the item;
generating, from the base model and based on one or more input parameters, a device-specific three-dimensional (3D) model of the item by removing at least one geometric primitive from the base model while preserving one or more of the visual elements, wherein the one or more visual elements comprise at least one projection boundary, at least one surface contour, or combinations thereof,
wherein the one or more input parameters comprise:
one or more device characteristics of the user device,
a desired number of geometric primitives, and
an indication to preserve the one or more visual elements, and

wherein the device-specific 3D model comprises:
the desired number of geometric primitives, and
the preserved one or more visual elements; and

transmitting, to the user device, a data object comprising the device-specific 3D model, wherein the data object comprises data for generating a binocular rendering of the device-specific 3D model.

2. The method of claim 1 , further comprising:
determining, based on image data associated with the plurality of images, physical attributes of the item, wherein the physical attributes comprise at least texture data of the item.

3. The method of claim 1 , wherein the geometric data comprises a geometric mesh of three-dimensional points determined using texture data.

4. The method of claim 1 , wherein the one or more device characteristics comprise a display resolution of the user device, a processing capability of the user device, or a display interface of the user device.

5. The method of claim 1 , wherein the request to view the item comprises a request to view a 3D model of the item.

6. The method of claim 1 , further comprising receiving user input comprising the one or more input parameters.

7. The method of claim 1 , wherein the binocular rendering of the device-specific 3D model is configured for presentation on at least one of: an augmented reality display device or a virtual reality display device.

8. A non-transitory, computer-readable storage medium storing instructions, that when executed by at least one processor, cause a system to perform operations comprising:
extracting points from each of a plurality of images depicting different two-dimensional orientations of an item, to generate geometric data comprising geometric primitives and visual elements of the item;
generating, using the geometric data, a base model of the item;
receiving, from a user device, a request to view the item;
generating, from the base model and based on one or more input parameters, a device-specific three-dimensional (3D) model of the item by removing at least one geometric primitive from the base model while preserving one or more of the visual elements, wherein the one or more visual elements comprise at least one projection boundary, at least one surface contour, or combinations thereof,
wherein the one or more input parameters comprise:
one or more device characteristics of the user device,
a desired number of geometric primitives, and
an indication to preserve the one or more visual elements, and

wherein the device-specific 3D model comprises:
the desired number of geometric primitives, and
the preserved one or more visual elements; and

transmitting, to the user device, a data object comprising the device-specific 3D model, wherein the data object comprises data for generating a binocular rendering of the device-specific 3D model.

9. The non-transitory, computer-readable storage medium of claim 8 , wherein the instructions, when executed by the at least one processor, causes the system to perform operations comprising:
determining, based on image data associated with the plurality of images, physical attributes of the item, wherein the physical attributes comprise at least texture data of the item.

10. The non-transitory, computer-readable storage medium of claim 8 , wherein the geometric data comprises a geometric mesh of three-dimensional points determined using texture data.

11. The non-transitory, computer-readable storage medium of claim 8 , wherein the one or more device characteristics comprise a display resolution of the user device, a processing capability of the user device, or a display interface of the user device.

12. The non-transitory, computer-readable storage medium of claim 8 , wherein the request to view the item comprises a request to view a 3D model of the item.

13. The non-transitory, computer-readable storage medium of claim 8 , wherein the instructions, when executed by the at least one processor, cause the system to perform operations comprising receiving user input comprising the one or more input parameters.

14. A system comprising:
at least one processor; and
memory storing instructions, that when executed by the at least one processor, cause the system to:
extract points from each of a plurality of images depicting different two-dimensional orientations of an item, to generate geometric data comprising geometric primitives and visual elements of the item;
generate, using the geometric data, a base model of the item;
receive, from a user device, a request to view the item;
generate, from the base model and based on one or more input parameters, a device-specific three-dimensional (3D) model of the item by removing at least one geometric primitive from the base model while preserving one or more of the visual elements, wherein the one or more visual elements comprise at least one projection boundary, at least one surface contour, or combinations thereof,
wherein the one or more input parameters comprise:
one or more device characteristics of the user device,
a desired number of geometric primitives, and
an indication to preserve the one or more visual elements, and

wherein the device-specific 3D model comprises:
the desired number of geometric primitives, and
the preserved one or more visual elements; and

transmit, to the user device, a data object comprising the device-specific 3D model, wherein the data object comprises data for generating a binocular rendering of the device-specific 3D model.

15. The system of claim 14 , wherein the instructions, when executed by the at least one processor, further cause the system to determine, based on image data associated with the plurality of images, physical attributes of the item, wherein the physical attributes comprise at least texture data of the item.

16. The system of claim 14 , wherein the geometric data comprises a geometric mesh of three-dimensional points determined using texture data.

17. The system of claim 14 , wherein the one or more device characteristics comprise a communication bandwidth between the user device and the system, a display resolution of the user device, a processing capability of the user device, or a display interface of the user device.

18. The system of claim 14 , wherein the request to view the item comprises a request to view a 3D model of the item.

Assignees

• Ebay Inc

Inventors

• Xie Jinrong
• Wang Qiaosong
• Piramuthu Robinson

Classifications

• G06T2219/2016

  Rotation, translation, scaling · CPC title

• G06T2219/004

  Annotating, labelling · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title

• G06T2200/08

  involving all processing steps from image acquisition to 3D model generation · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title