Calibrating a machine vision camera

Patent metadata

Field	Value
Publication number	US-11436756-B2
Application number	US-201816228688-A
Country	US
Kind code	B2
Filing date	Dec 20, 2018
Priority date	Dec 20, 2018
Publication date	Sep 6, 2022
Grant date	Sep 6, 2022

Abstract

Examples are disclosed that relate to a camera model for a machine vision application. One example provides instructions executable to receive image data obtained by an image sensor of a camera, the image data capturing a calibration pattern comprising a plurality of calibration features, for each of one or more imaged calibration features in the image data, determine an object space location of the imaged calibration feature, and determine a distance between the object space location and a corresponding ray of a camera model, the camera model defining a plurality of rays that each represent a relationship of an image space location on the image sensor to object space. The instructions are further executable to determine a value of a cost function based on the distances, adjust the camera model until the cost function meets a target condition, and use the camera model in a machine vision application.

First claim

The invention claimed is:

1. A computing device, comprising:
a logic machine; and
a storage machine comprising instructions executable by the logic machine to
receive image data obtained by an image sensor of a camera, the image data capturing a calibration pattern comprising a plurality of calibration features
for each of a plurality of imaged calibration features in the image data,
determine an object space location of the imaged calibration feature;
determine a location of the imaged calibration feature in image space; and
determine a distance between the object space location of the imaged calibration feature and a corresponding ray of a camera model corresponding to the location of the imaged calibration feature in image space, the camera model defining a plurality of rays that each represents a relationship of an image space location on the image sensor to object space, and each ray comprising a first coordinate at a first distance from the image sensor and a second coordinate at a second distance from the image sensor;

determine a value of a cost function based on the distances determined for the plurality of imaged calibration features;
for each imaged calibration feature, fit the corresponding ray to the object space location of the imaged calibration feature by varying one or more of the first coordinate and the second coordinate for the corresponding ray to adjust the camera model until the cost function meets a target condition; and
use the camera model in a machine vision application.

2. The computing device of claim 1 , wherein the image data comprises a plurality of images.

3. The computing device of claim 1 , wherein the camera is a part of a non-central imaging system.

4. The computing device of claim 3 , wherein the camera is positioned to receive light refracted through a lens.

5. The computing device of claim 1 , wherein the instructions executable to use the camera model in the machine vision application are implemented by an eye-tracking system.

6. The computing device of claim 1 , wherein, for each ray of the plurality of rays, the first coordinate comprises a first (x,y) coordinate and the second coordinate comprises a second (x,y) coordinate.

7. The computing device of claim 1 , further comprising instructions executable to recalibrate the camera model after one or more of damage to the computing device and replacement of a part of the computing device.

8. The computing device of claim 1 , wherein the instructions executable to adjust the camera model are executable to modify the corresponding ray such that the distance between the object space location of the imaged calibration feature and the corresponding ray is reduced.

9. The computing device of claim 1 , wherein the relationship of the image space location to object space is encoded by one or more splines.

10. The computing device of claim 9 , wherein the instructions executable to adjust the camera model are executable to adjust one or more parameters of the one or more splines.

11. The computing device of claim 1 , wherein the object space location of the imaged calibration feature is determined based upon a transformation from a calibration pattern coordinate system to a coordinate system of the camera.

12. At a computing device, a method, comprising:
receiving image data obtained by an image sensor of a camera, the image data capturing a calibration pattern comprising a plurality of calibration features;
for each of a plurality of imaged calibration features in the image data,
determining an object space location of the imaged calibration feature,
determining a location of the imaged calibration feature in image space, and
determining a distance between the object space location of the imaged calibration feature and a corresponding ray of a camera model corresponding to the location of the imaged calibration feature in image space, the camera model defining a plurality of rays that each represents a relationship of an image space location on the image sensor to object space, and each ray comprising a first coordinate at a first distance from the image sensor and a second coordinate at a second distance from the image sensor;

determining a value of a cost function based on the distances determined for the plurality of imaged calibration features; and
for each imaged calibration feature, fitting the corresponding ray to the object space location of the imaged calibration feature by varying one or more of the first coordinate and the second coordinate for the corresponding ray to adjust the camera model until the cost function meets a target condition.

13. The method of claim 12 , wherein the camera model is used by an eye-tracking system.

14. The method of claim 12 , wherein, for each ray of the plurality of rays, the first coordinate comprises a first (x,y) coordinate and the second coordinate comprises a second (x,y) coordinate.

15. The method of claim 12 , further comprising readjusting the camera model based upon a recalibration pattern after one or more of damage to the computing device and replacement of a part of the computing device.

16. The method of claim 12 , wherein adjusting the camera model comprises modifying the corresponding ray such that the distance between the object space location of the imaged calibration feature and the corresponding ray is reduced.

17. The method of claim 12 , wherein the relationship of the image space location to object space is encoded by one or more splines.

18. The method of claim 17 , wherein adjusting the camera model comprises adjusting the camera model on a subpixel scale.

19. A head-mounted display device, comprising:
a camera comprising an image sensor;
a logic machine; and
a storage machine comprising instructions executable by the logic machine to
via the camera, obtain image data capturing a calibration pattern comprising a plurality of calibration features;
for each of a plurality of imaged calibration features in the image data,
determine an object space location of the imaged calibration feature;
determine a location of the imaged calibration feature in image space;
determine a distance between the object space location of the imaged calibration feature and a corresponding ray of a camera model corresponding to the location of the imaged calibration feature in image space, the camera model defining a plurality of rays that each represent a relationship of an image space location on the image sensor to object space, and each ray comprising a first coordinate at a first distance from the image sensor and a second coordinate at a second distance from the image sensor;
determine a value of a cost function based on the distances determined for the plurality of imaged calibration features;
for each imaged calibration feature, fit the corresponding ray to the object space location of the imaged calibration feature by varying one or more of the first coordinate and the second coordinate for the corresponding ray to adjust the camera model until the cost function meets a target condition;
use the camera model in a machine vision application; and
recalibrate the camera model after one or more of damage to the head-mounted display device and replacement of a part of the head-mounted display device.

20. The head-mounted display device of claim 19 , wherein the instructions are executable to recalibrate the camera model based on capturing images of a recalibration pattern and fitting one or more rays of the camera model to observations of calibration fea

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Fitzgibbon Andrew William
• Khapko Taras
• Jovanovic Vuk
• Panjevic Filip
• Carapic Vladimir
• Mojasevic Jelena

Classifications

• G06T7/73

  using feature-based methods · CPC title

• G06T7/80 Primary

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title