Camera calibration

Patent metadata

Field	Value
Publication number	US-12197208-B2
Application number	US-202117515855-A
Country	US
Kind code	B2
Filing date	Nov 1, 2021
Priority date	Nov 1, 2021
Publication date	Jan 14, 2025
Grant date	Jan 14, 2025

Abstract

A first plurality of center points of first two-dimensional bounding boxes corresponding to a vehicle occurring in a first plurality of images acquired by a first camera can be determined. A second plurality of center points of second two-dimensional bounding boxes corresponding to the vehicle occurring in a second plurality of images acquired by a second camera can also be determined. A plurality of non-linear equations based on the locations of the first and second pluralities of center points and first and second camera parameters corresponding to the first and second cameras can be determined. The plurality of non-linear equations can be solved simultaneously for the locations of the vehicle with respect to the first and second cameras and the six degree of freedom pose of the second camera with respect to the first camera.

First claim

The invention claimed is:

1. A computer, comprising:
a processor; and
a memory, the memory including instructions executable by the processor to:
determine a first plurality of center points of first two-dimensional bounding boxes corresponding to locations of a vehicle occurring in a first plurality of images acquired by a first camera;
determine a second plurality of center points of second two-dimensional bounding boxes corresponding to the locations of the vehicle occurring in a second plurality of images acquired by a second camera;
determine a plurality of non-linear equations based on respective locations of the first and second pluralities of center points and first and second camera locations including camera parameters corresponding to the first and second cameras;
simultaneously solve the plurality of non-linear equations for the locations of the vehicle with respect to the first and second cameras and a six degree of freedom pose of the second camera with respect to the first camera;
determine real-world coordinates of the six degree of freedom pose of the second camera based on real-world coordinates of a six degree of freedom pose of the first camera; and
transmit the real-world coordinates of the six degree of freedom pose for the first camera, and the real-world coordinates of the six degree of freedom pose for the second camera to a second computer included in a vehicle to control the motion of the vehicle.

2. The computer of claim 1 , wherein first and second camera parameters include the six degree of freedom poses of the first and second cameras.

3. The computer of claim 1 , wherein the real-world coordinates of the first camera are determined by locating the first camera using lidar data.

4. The computer of claim 1 , the instructions including further instructions to determine the first and second plurality of center points based on first and second bounding boxes by inputting the first and second pluralities of images to a convolutional neural network.

5. The computer of claim 1 , the instructions including further instructions to solve the plurality of non-linear equations using Gauss-Newton iteration.

6. The computer of claim 5 , wherein solving the plurality of non-linear equations using Gauss-Newton iteration includes determining a Jacobian matrix of partial derivatives.

7. The computer of claim 1 , the instructions including further instructions to solve the non-linear equations using a Levenberg-Marquardt algorithm.

8. The computer of claim 1 , wherein simultaneously solving the plurality of non-linear equations for the locations of the vehicle with respect to the first and second cameras and the six degree of freedom pose of the second camera with respect to the first camera includes constraining the first and second two-dimensional bounding boxes to a plane.

9. The computer of claim 1 , wherein simultaneously solving the plurality of non-linear equations for the locations of the vehicle with respect to the first and second cameras and the six degree of freedom pose of the second camera with respect to the first camera includes constraining the locations of the vehicle based on lidar data.

10. The computer of claim 1 , wherein simultaneously solving the plurality of non-linear equations for the locations of the vehicle with respect to the first and second cameras and the six degree of freedom pose of the second camera with respect to the first camera includes constraining the locations of the vehicle based on one or more of global positioning system data, inertial measurement unit data and visual odometry data.

11. The computer of claim 1 , wherein simultaneously solving the plurality of non-linear equations for the locations of the vehicle with respect to the first and second cameras and the six degree of freedom pose of the second camera with respect to the first camera includes constraining the locations of the vehicle based on map data.

12. The computer of claim 1 , wherein simultaneously solving the plurality of non-linear equations for the locations of the vehicle with respect to the first and second cameras and the six degree of freedom pose of the second camera with respect to the first camera includes constraining the locations of the vehicle based on center points determined based on three-dimensional bounding boxes.

13. A method, comprising:
determining a first plurality of center points of first two-dimensional bounding boxes corresponding to a vehicle occurring in a first plurality of images acquired by a first camera;
determining a second plurality of center points of second two-dimensional bounding boxes corresponding to the vehicle occurring in a second plurality of images acquired by a second camera;
determining a plurality of non-linear equations based on respective locations of the first and second pluralities of center points and first and second camera locations including camera parameters corresponding to the first and second cameras;
simultaneously solving the plurality of non-linear equations for the locations of the vehicle with respect to the first and second cameras and a six degree of freedom pose of the second camera with respect to the first camera;
determining real-world coordinates of the six degree of freedom pose of the second camera based on real-world coordinates of a six degree of freedom pose of the first camera; and
transmit the real-world coordinates of the six degree of freedom pose for the first camera, and the real-world coordinates of the six degree of freedom pose for the second camera to a second computer included in a vehicle to control the motion of the vehicle.

14. The method of claim 13 , wherein first and second camera parameters include the six degree of freedom poses of the first and second cameras.

15. The method of claim 13 , wherein the real-world coordinates of the first camera are determined by locating the first camera using lidar data.

16. The method of claim 13 , further comprising determining the first and second plurality of center points based on first and second bounding boxes by inputting the first and second pluralities of images to a convolutional neural network.

17. The method of claim 13 , further comprising solving the plurality of non-linear equations using Gauss-Newton iteration.

18. The method of claim 17 , wherein solving the plurality of non-linear equations using Gauss-Newton iteration includes determining a Jacobian matrix of partial derivatives.

19. The method of claim 18 , wherein solving the plurality of non-linear equations using Gauss-Newton iteration includes determining a Jacobian matrix of partial derivatives.

20. The method of claim 13 , the instructions including further instructions to solve the non-linear equations using a Levenberg-Marquardt algorithm.

Assignees

• Ford Global Tech Llc

Inventors

• Chakravarty Punarjay
• Shrivastava Shubham
• Ghadge Bhushan
• Parchami Mostafa
• Pandey Gaurav

Classifications

• G05D1/2235

  involving the operator tracking the vehicle by direct line of sight · CPC title

• G06T2207/30244

  Camera pose · CPC title

• G06T2210/12

  Bounding box · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title