Real-time image undistortion for incremental 3D reconstruction

Patent metadata

Field	Value
Publication number	US-10269148-B2
Application number	US-201715583074-A
Country	US
Kind code	B2
Filing date	May 1, 2017
Priority date	May 1, 2017
Publication date	Apr 23, 2019
Grant date	Apr 23, 2019

Abstract

A system provides image undistortion in 3D reconstruction. The system receives an image produced by a sensor, and determines whether correction values are cached for the sensor, where each correction value is configured to place a corresponding pixel into a corrected location. When there are no cached correction values, the system calculates correction values for pixels in the image, generates a correction grid for the image including vertices corresponding to texture coordinates from the image, where each vertex in the correction grid includes a corresponding correction value, partitions the correction grid into partitioned grids, and caches the partitioned grids. The system then renders the image using the partitioned grids.

First claim

What is claimed is:

1. A method for image undistortion in three-dimensional (3D) reconstruction, comprising:
receiving an image produced by a sensor;
determining whether correction values are cached for the sensor, wherein each correction value is configured to place a corresponding pixel into a corrected location;
when there are no cached correction values, calculating correction values for pixels in the image, generating a correction grid for the image including vertices corresponding to texture coordinates from the image, wherein each vertex in the correction grid includes a corresponding correction value, partitioning the correction grid into partitioned grids, and caching the partitioned grids; and
rendering the image using the partitioned grids.

2. The method of claim 1 , wherein the correction values are calculated based on intrinsic and lens distortion parameters describing the sensor.

3. The method of claim 1 , wherein each correction value is independently calculated in a threaded parallel operation on a central processing unit (CPU) or a graphics processing unit (GPU).

4. The method of claim 1 , wherein the partitioned grids are contained in vertex buffers allocated in a memory of a graphics processing unit (GPU).

5. The method of claim 4 , wherein a vertex buffer corresponding to a partitioned grid is rendered using the GPU as an orthographic 3D rendering in a render loop.

6. The method of claim 5 , wherein, at each render cycle, a 3D scene is drawn, and then a respective partitioned grid is drawn.

7. The method of claim 6 , wherein the render cycle is repeated once for each partitioned grid.

8. The method of claim 1 , wherein the calculating of the correction values, the generating of the correction grid, the partitioning of the correction grid, and the caching of the partitioned grids are performed only once for undistortion of images produced by the sensor.

9. The method of claim 1 , wherein each partitioned grid is a rectangular segment of the correction grid.

10. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processor, cause the processor to provide image undistortion in three-dimensional (3D) reconstruction, the processor:
receiving an image produced by a sensor;
determining whether correction values are cached for the sensor, wherein each correction value is configured to place a corresponding pixel into a corrected location;
when there are no cached correction values, calculating correction values for pixels in the image, generating a correction grid for the image including vertices corresponding to texture coordinates from the image, wherein each vertex in the correction grid includes a corresponding correction value, partitioning the correction grid into partitioned grids, and caching the partitioned grids; and
rendering the image using the partitioned grids.

11. The computer readable medium of claim 10 , wherein the correction values are calculated based on intrinsic and lens distortion parameters describing the sensor.

12. The computer readable medium of claim 10 , wherein each correction value is independently calculated in a threaded parallel operation on a central processing unit (CPU) or a graphics processing unit (GPU).

13. The computer readable medium of claim 10 , wherein the partitioned grids are contained in vertex buffers allocated in a memory of a graphics processing unit (GPU).

14. The computer readable medium of claim 13 , wherein a vertex buffer corresponding to a partitioned grid is rendered using the GPU as an orthographic 3D rendering in a render loop.

15. The computer readable medium of claim 14 , wherein, at each render cycle, a 3D scene is drawn, and then a respective partitioned grid is drawn.

16. The computer readable medium of claim 15 , wherein the render cycle is repeated once for each partitioned grid.

17. The computer readable medium of claim 10 , wherein the calculating of the correction values, the generating of the correction grid, the partitioning of the correction grid, and the caching of the partitioned grids are performed only once for undistortion of images produced by the sensor.

18. The computer readable medium of claim 10 , wherein each partitioned grid is a rectangular segment of the correction grid.

19. A system for providing image undistortion in three-dimensional (3D) reconstruction, the system comprising:
a processor; and
a memory coupled with the processor and storing instructions that, when executed by the processor, cause the processor to:
receive an image produced by a sensor;
determine whether correction values are cached for the sensor, wherein each correction value is configured to place a corresponding pixel into a corrected location;
when there are no cached correction values, calculate correction values for pixels in the image, generate a correction grid for the image including vertices corresponding to texture coordinates from the image, wherein each vertex in the correction grid includes a corresponding correction value, partition the correction grid into partitioned grids, and cache the partitioned grids; and
render the image using the partitioned grids.

20. The system of claim 19 , wherein the correction values are calculated based on intrinsic and lens distortion parameters describing the sensor.

Assignees

• Lockheed Corp

Inventors

• Jones Michael

Classifications

• G06T12/10 Primary

  Image preprocessing, e.g. calibration, positioning of sources or scatter correction · CPC title

• G06T11/005 Primary

  Physics · mapped topic

• G06T15/005

  General purpose rendering architectures · CPC title

• G06T2207/10032

  Satellite or aerial image; Remote sensing · CPC title

• G06T5/002

  Physics · mapped topic