Systems and methods for fisheye camera calibration and bird's-eye-view image generation in a simulation environment

Patent metadata

Field	Value
Publication number	US-11812153-B2
Application number	US-202217687794-A
Country	US
Kind code	B2
Filing date	Mar 7, 2022
Priority date	Mar 7, 2022
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Systems and methods for fisheye camera calibration and BEV image generation in a simulation environment. This fisheye camera calibration enables the extrinsic and intrinsic parameters of the fisheye camera to be computed in the simulation environment, where data is readily available, collectible, and manipulatable. Given a surround vision system, with multiple fisheye cameras disposed around a vehicle, and these extrinsic and intrinsic parameters, undistorted and BEV images of the surroundings of the vehicle can be generated in the simulated environment, for simulated fisheye camera testing and validation, which may then be extrapolated to real-world fisheye camera testing and validation, as appropriate. Because the simulation tool can be used to create and readily manipulate the simulated fisheye camera, the vehicle, its surroundings, obstacles, targets, markers, and the like, the entire calibration and image generation process is streamlined and may be automated.

First claim

What is claimed is:

1. A simulated camera system, comprising:
memory storing instructions executed by a processor to generate a simulated camera in a simulated environment and obtain a distorted image using the simulated camera;
memory storing instructions executed by the processor to calibrate the simulated camera and obtain intrinsic parameters of the simulated camera by successive imaging of a repositioned target/marker generated in the simulated environment using the simulated camera; and
memory storing instructions executed by the processor to generate an undistorted image from the distorted image using the intrinsic parameters of the simulated camera.

2. The simulated camera system of claim 1 , further comprising:
memory storing instructions executed by the processor to determine coordinates of the simulated camera in the simulated environment and obtain extrinsic parameters of the simulated camera; and
memory storing instructions executed by the processor to generate a perspective-shifted image from the undistorted image using the extrinsic parameters of the simulated camera.

3. The simulated camera system of claim 2 , further comprising memory storing instructions executed by the processor to obtain a plurality of distorted images using the simulated camera, generate a plurality of undistorted images from the plurality of distorted images, generate a plurality of perspective-shifted images from the plurality of undistorted images, and stitch the plurality of perspective-shifted images together.

4. The simulated camera system of claim 2 , wherein the perspective-shifted image comprises a bird's-eye-view image.

5. The simulated camera system of claim 1 , wherein the simulated camera comprises a simulated fisheye camera and the distorted image comprises a fisheye image.

6. The simulated camera system of claim 1 , further comprising memory storing instructions executed by the processor to use the intrinsic parameters of the simulated camera to generate an undistortion and rectification transformation map that is used to generate the undistorted image from the distorted image.

7. The simulated camera system of claim 1 , further comprising memory storing instructions executed by the processor to iteratively calibrate the simulated camera using an artificial intelligence algorithm.

8. A simulated camera method, comprising:
generating a simulated camera in a simulated environment and obtaining a distorted image using the simulated camera;
calibrating the simulated camera and obtaining intrinsic parameters of the simulated camera by successive imaging of a repositioned target/marker generated in the simulated environment using the simulated camera; and
generating an undistorted image from the distorted image using the intrinsic parameters of the simulated camera.

9. The simulated camera method of claim 8 , further comprising:
determining coordinates of the simulated camera in the simulated environment and obtaining extrinsic parameters of the simulated camera; and
generating a perspective-shifted image from the undistorted image using the extrinsic parameters of the simulated camera.

10. The simulated camera method of claim 9 , further comprising obtaining a plurality of distorted images using the simulated camera, generating a plurality of undistorted images from the plurality of distorted images, generating a plurality of perspective-shifted images from the plurality of undistorted images, and stitching the plurality of perspective-shifted images together.

11. The simulated camera method of claim 9 , wherein the perspective-shifted image comprises a bird's-eye-view image.

12. The simulated camera method of claim 8 , wherein the simulated camera comprises a simulated fisheye camera and the distorted image comprises a fisheye image.

13. The simulated camera method of claim 8 , further comprising using the intrinsic parameters of the simulated camera to generate an undistortion and rectification transformation map that is used to generate the undistorted image from the distorted image.

14. The simulated camera method of claim 8 , further comprising iteratively calibrating the simulated camera using an artificial intelligence algorithm.

15. A non-transitory computer-readable medium comprising instructions stored in a memory and executed by a processor to carry out simulated camera steps comprising:
generating a simulated camera in a simulated environment and obtaining a distorted image using the simulated camera;
calibrating the simulated camera and obtaining intrinsic parameters of the simulated camera by successive imaging of a repositioned target/marker generated in the simulated environment using the simulated camera; and
generating an undistorted image from the distorted image using the intrinsic parameters of the simulated camera.

16. The non-transitory computer-readable medium of claim 15 , the steps further comprising:
determining coordinates of the simulated camera in the simulated environment and obtaining extrinsic parameters of the simulated camera; and
generating a perspective-shifted image from the undistorted image using the extrinsic parameters of the simulated camera.

17. The non-transitory computer-readable medium of claim 16 , the steps further comprising obtaining a plurality of distorted images using the simulated camera, generating a plurality of undistorted images from the plurality of distorted images, generating a plurality of perspective-shifted images from the plurality of undistorted images, and stitching the plurality of perspective-shifted images together.

18. The non-transitory computer-readable medium of claim 16 , wherein the perspective-shifted image comprises a bird's-eye-view image.

19. The non-transitory computer-readable medium of claim 15 , wherein the simulated camera comprises a simulated fisheye camera and the distorted image comprises a fisheye image.

20. The non-transitory computer-readable medium of claim 15 , the steps further comprising using the intrinsic parameters of the simulated camera to generate an undistortion and rectification transformation map that is used to generate the undistorted image from the distorted image.

Assignees

• Volvo Car Corp

Inventors

• Liu Zhipeng
• Suga Chihiro
• Ding Sihao

Classifications

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• B60R1/28 Primary

  with an adjustable field of view · CPC title

• B60R2300/101

  using cameras with adjustable capturing direction · CPC title

• B60R2300/30

  characterised by the type of image processing · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title