Methods and systems for simulating image capture of imaging device

Patent metadata

Field	Value
Publication number	US-12499610-B2
Application number	US-202218562395-A
Country	US
Kind code	B2
Filing date	May 29, 2022
Priority date	Aug 2, 2021
Publication date	Dec 16, 2025
Grant date	Dec 16, 2025

Abstract

Methods, systems, and computer program products simulate image-capture of a scene at having least one object of interest (OOI) by a camera. A plurality of rendered images is obtained and includes: i) a first image, at a first resolution, that is an image of the scene and excludes the OOI, ii) a second image, at a second resolution higher than the first resolution, that is an image of the OOI, iii) a third image, at the second resolution, that is an image of a region that bounds the OOI, wherein the OOI at least partially affects one or more other objects in the region, and iv) a fourth image, at the second resolution, wherein for each pixel of the fourth image, a pixel value of the pixel indicates a contribution of the OOI in the pixel. The images are processed to produce a high-resolution image of the OOI.

First claim

What is claimed is:

1 . A method for simulating image-capture of a scene by a camera, the scene containing at least one object of interest (OOI), the method comprising:
a) obtaining a plurality of images each having an associated resolution, the plurality of images including:
i) a first image, at a first resolution, that is an image of the scene and excludes the OOI,
ii) a second image, at a second resolution higher than the first resolution, that is an image of the OOI,
iii) a third image, at the second resolution, that is an image of a region that bounds the OOI, wherein the OOI at least partially affects one or more other objects in the region, and
iv) a fourth image, at the second resolution, wherein for each pixel of the fourth image, a pixel value of the pixel indicates a contribution of the OOI in the pixel;
b) multiplying together the second image and the fourth image to produce a product image; c) reducing the resolution of the product image to the first resolution to produce a reduced resolution product image; d) reducing the resolution of each of the third image and the fourth image to the first resolution to produce a reduced resolution third image and a reduced resolution fourth image; and e) combining together the reduced resolution product image and an intermediate image to produce an output image, the intermediate image being a weighted combination of the first image and the reduced resolution third image produced using weights based on the reduced resolution fourth image.

2 . The method of claim 1 , wherein reducing the resolution of the product image to the first resolution includes: downsampling and blurring the product image to produce a blurred-downsampled image at an interim resolution, and downsampling the blurred-downsampled image to the first resolution.

3 . The method of claim 1 , wherein the intermediate image is produced by: i) applying weights based on the reduced resolution fourth image to the first image to produce a weighted version of the first image, ii) applying weights based on the reduced resolution fourth image to the reduced resolution third image to produce a weighted version of the reduced resolution third image, and iii) adding together the weighted version first image and the weighted version of the reduced resolution third image to produce the weighted image.

4 . The method of claim 1 , wherein combining together the reduced resolution product image and the weighted image includes: adding the reduced resolution product image to the weighted image to produce the output image.

5 . The method of claim 1 , wherein the fourth image is generated based at least in part on each of the second image and the third image.

6 . The method of claim 1 , wherein the region includes one or more other objects in addition to the OOI.

7 . The method of claim 6 , wherein the second image excludes all of the one or more other objects.

8 . The method of claim 1 , wherein the second resolution is received as input from a user.

9 . The method of claim 1 , wherein the method is implemented on at least one processor using a graphics processing pipeline.

10 . The method of claim 1 , wherein the scene includes a plurality of OOIs, and wherein a) through e) are performed for each of the OOIs.

11 . A system for simulating image-capture of a scene by a camera, the scene containing at least one object of interest (OOI), the system comprising:
a) a processing unit including at least one processor in communication with at least one storage medium configured to:
i) obtain a plurality of images each having an associated resolution, the plurality of images including:
1) a first image, at a first resolution, that is an image of the scene and excludes the OOI,
2) a second image, at a second resolution higher than the first resolution, that is an image of the OOI,
3) a third image, at the second resolution, that is an image of a region that bounds the OOI, wherein the OOI at least partially affects one or more other objects in the region, and
4) a fourth image, at the second resolution, wherein for each pixel of the fourth image, a pixel value of the pixel indicates a contribution of the OOI in the pixel,

ii) multiply together the second image and the fourth image to produce a product image,
iii) reduce the resolution of the product image to the first resolution to produce a reduced resolution product image,
iv) reduce the resolution of each of the third image and the fourth image to the first resolution to produce a reduced resolution third image and a reduced resolution fourth image, and
v) combine together the reduced resolution product image and an intermediate image to produce an output image, the intermediate image being a weighted combination of the first image and the reduced resolution third image produced using weights based on the reduced resolution fourth image.

12 . The system of claim 11 , wherein the processing unit is configured to reduce the resolution of the product image to the first resolution by downsampling and blurring the product image to produce a blurred-downsampled image at an interim resolution, and downsampling the blurred-downsampled image to the first resolution.

13 . The system of claim 11 , wherein the processing unit is configured to produce the intermediate image by: i) applying weights based on the reduced resolution fourth image to the first image to produce a weighted version of the first image, ii) applying weights based on the reduced resolution fourth image to the reduced resolution third image to produce a weighted version of the reduced resolution third image, and iii) adding together the weighted version first image and the weighted version of the reduced resolution third image to produce the weighted image.

14 . The system of claim 11 , wherein the processing unit is configured to combine together the reduced resolution product image and the weighted image by: adding the reduced resolution product image to the weighted image to produce the output image.

15 . The system of claim 11 , wherein the processing unit is further configured to generate the fourth image based at least in part on each of the second image and the third image.

16 . The system of claim 11 , wherein the region includes one or more other objects in addition to the OOI.

17 . The system of claim 16 , wherein the second image excludes all of the one or more other objects.

18 . The system of claim 11 , wherein the processing unit is further configured to receive the second resolution as input from a user.

19 . The system of claim 11 , further comprising:
b) an input interface in communication with the processing unit for providing at least the second resolution as input to the processing unit; and c) an output interface in communication with the processing unit for providing at least the output image as output from the system.

20 . The system of claim 11 , wherein the processing unit is configured as a graphical processing unit that implements a graphics processing pipeline.

21 . A computer usable non-transitory storage medium having a computer program embodied thereon for causing a suitably programmed system to produce an image thereby simulating image-capture of a scene by a camera, the scene containing at least one object of interest (OOI), by performing the following steps when such program is executed on the system, the steps comprising:
a) obtaining a plurality of images each having an associated resolution, the plurali

Assignees

• Rafael Advanced Defense Systems Ltd

Inventors

• Vardi Yair

Classifications

• G06T2207/20221

  Image fusion; Image merging · CPC title

• G06T3/40 Primary

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06T5/70

  Denoising; Smoothing · CPC title

• G06T2200/28

  involving image processing hardware · CPC title

• G06T2200/21

  involving computational photography · CPC title