Arbitrary view generation

Patent metadata

Field	Value
Publication number	US-12561894-B2
Application number	US-202117219718-A
Country	US
Kind code	B2
Filing date	Mar 31, 2021
Priority date	Mar 25, 2016
Publication date	Feb 24, 2026
Grant date	Feb 24, 2026

Abstract

Techniques for generating an arbitrary view of an asset are disclosed. In some embodiments, arbitrary view generation includes storing a set of images of an asset, wherein each image comprising at least a subset of the set of images is rendered from a three-dimensional model of the asset, and generating an image comprising an arbitrary perspective of the asset at least in part by populating the image comprising the arbitrary perspective with pixels harvested from one or more images comprising the set of images.

First claim

What is claimed is:

1 . A system, comprising:
a database configured to store an existing set of images of an asset, wherein each image comprising at least a subset of the existing set of images is rendered from a three-dimensional model of the asset; and a processor configured to generate an output image comprising an arbitrary perspective of the asset that comprises a different camera perspective than any image in the existing set of images of the asset at least in part by populating the output image comprising the arbitrary perspective with pixels harvested from one or more images comprising the existing set of images.

2 . The system of claim 1 , wherein to generate the output image comprising the arbitrary perspective comprises to normalize the one or more images to the arbitrary perspective prior to harvesting pixels.

3 . The system of claim 1 , wherein to generate the output image comprising the arbitrary perspective comprises to transform the one or more images from other perspectives to the arbitrary perspective prior to harvesting pixels.

4 . The system of claim 1 , wherein the one or more images comprise at least some missing pixels resulting from normalization or transformation to the arbitrary perspective from other perspectives.

5 . The system of claim 1 , wherein to generate the output image comprising the arbitrary perspective comprises to interpolate one or more pixels of the output image comprising the arbitrary perspective.

6 . The system of claim 1 , wherein the three-dimensional model of the asset comprises a three-dimensional polygon mesh model of the asset.

7 . The system of claim 1 , wherein the three-dimensional model of the asset is generated based on photographs of the asset captured using an imaging device.

8 . The system of claim 1 , wherein the three-dimensional model of the asset is generated based on point cloud data associated with the asset collected using a scanner.

9 . The system of claim 1 , wherein the three-dimensional model of the asset is generated based on metadata captured while imaging or scanning the asset.

10 . The system of claim 1 , wherein at least one image of the subset is rendered using a scanline or rasterization rendering technique.

11 . The system of claim 1 , wherein at least one image of the subset is rendered using ray tracing.

12 . The system of claim 1 , wherein the images comprising the existing set of images comprise different perspectives.

13 . The system of claim 1 , wherein the images comprising the existing set of images comprise perspectives different than the arbitrary perspective.

14 . The system of claim 1 , wherein the existing set of images includes one or more orthographic views of the asset.

15 . The system of claim 1 , wherein at least one image of the existing set of images comprises metadata including camera information.

16 . The system of claim 1 , wherein at least one image of the existing set of images comprises camera characteristics including one or more of: position, orientation, rotation, angle, depth, focal length, aperture, and zoom level.

17 . The system of claim 1 , wherein at least one image of the existing set of images comprises metadata including relative asset and camera location and orientation.

18 . The system of claim 1 , wherein at least one image of the existing set of images comprises metadata including lighting information.

19 . The system of claim 1 , wherein at least one image of the existing set of images comprises pixel metadata including relative pixel position coordinates.

20 . The system of claim 1 , wherein at least one image of the existing set of images comprises pixel metadata including surface normal vectors.

21 . The system of claim 1 , wherein at least one image of the existing set of images comprises pixel metadata including texture mapping coordinates.

22 . The system of claim 1 , wherein at least one image of the existing set of images comprises pixel metadata encoded in one or more of the red, green, and blue (RGB) channels of the pixels.

23 . The system of claim 1 , wherein the asset comprises a scene comprising one or more objects.

24 . The system of claim 1 , wherein the asset comprises a scene comprising a rich virtual environment.

25 . The system of claim 1 , wherein the existing set of images comprises photographs, photorealistic renderings, or both.

26 . The system of claim 1 , wherein the existing set of images is associated with a motion capture time slice.

27 . The system of claim 1 , wherein the generated output image comprising the arbitrary perspective comprises a still image or a frame of a video sequence.

28 . The system of claim 1 , wherein the system is associated with a gaming application, a virtual reality application, or a computer-generated imagery (CGI) application.

29 . A method, comprising:
storing an existing set of images of an asset, wherein each image comprising at least a subset of the existing set of images is rendered from a three-dimensional model of the asset; and generating an output image comprising an arbitrary perspective of the asset that comprises a different camera perspective than any image in the existing set of images of the asset at least in part by populating the output image comprising the arbitrary perspective with pixels harvested from one or more images comprising the existing set of images.

30 . The method of claim 29 , wherein generating the output image comprising the arbitrary perspective comprises normalizing the one or more images to the arbitrary perspective prior to harvesting pixels.

31 . The method of claim 29 , wherein generating the output image comprising the arbitrary perspective comprises transforming the one or more images from other perspectives to the arbitrary perspective prior to harvesting pixels.

32 . The method of claim 29 , wherein the one or more images comprise at least some missing pixels resulting from normalization or transformation to the arbitrary perspective from other perspectives.

33 . The method of claim 29 , wherein generating the output image comprising the arbitrary perspective comprises interpolating one or more pixels of the output image comprising the arbitrary perspective.

34 . The method of claim 29 , wherein the three-dimensional model of the asset comprises a three-dimensional polygon mesh model of the asset.

35 . The method of claim 29 , wherein the three-dimensional model of the asset is generated based on photographs of the asset captured using an imaging device.

36 . The method of claim 29 , wherein the three-dimensional model of the asset is generated based on point cloud data associated with the asset collected using a scanner.

37 . The method of claim 29 , wherein the three-dimensional model of the asset is generated based on metadata captured while imaging or scanning the asset.

38 . The method of claim 29 , wherein at least one image of the subset is rendered using a scanline or rasterization rendering technique.

39 . The method of claim 29 , wherein at least one image of the subset is rendered using ray tracing.

40 . The method o

Assignees

• Outward Inc

Inventors

• Chui Clarence
• Parmar Manu

Classifications

• G06F16/58

  Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually · CPC title

• G06T5/50

  using two or more images, e.g. averaging or subtraction · CPC title

• G06T7/32

  using correlation-based methods · CPC title

• G06T15/205 Primary

  Image-based rendering · CPC title