Calibrating devices by selecting images having a target having fiducial features

Patent metadata

Field	Value
Publication number	US-9338447-B1
Application number	US-201213419750-A
Country	US
Kind code	B1
Filing date	Mar 14, 2012
Priority date	Mar 14, 2012
Publication date	May 10, 2016
Grant date	May 10, 2016

Abstract

Devices and techniques are described for automatically calibrating a device such as a camera, projector, or both using a video stream. The device undergoing calibration is coupled to a computing device and configured to acquire video comprising a plurality of images. These images, at least some of the time, include targets having known characteristics present in an environment. From these acquired images calibration data comprising parameters such as intrinsic parameters, extrinsic parameters, or both, may be determined. The calibration data may be embedded or otherwise associated with the video stream as calibration matrix metadata.

First claim

What is claimed is:

1. A system comprising:
a camera configured to acquire video of an environment, wherein:
the environment includes a first portion that includes a target that presents a fiducial feature and a second portion that does not include the target, the first portion of the environment differing from the second portion of the environment;
the video comprises a plurality of images, at least one image depicting the target that presents the fiducial feature and at least one other image not depicting the target that presents the fiducial feature; and

a calibration module coupled to the camera and configured to:
select, from the plurality of images, images that depict the target that presents the fiducial feature;
determine, in each of the selected images, an observed location of the fiducial feature depicted in the respective selected image;
generate calibration data based at least in part on the observed locations of the fiducial feature in the selected images;
generate calibration matrix metadata based at least in part on the calibration data, the calibration matrix metadata comprising metadata for association with the video; and
associate at least a portion of the calibration matrix metadata with the video.

2. The system of claim 1 , wherein the associating comprises storing the at least a portion of the calibration matrix metadata in association with the video or modifying at least a portion of the video using the calibration matrix metadata.

3. The system of claim 1 , wherein the calibration module is further configured to continue generating calibration data until a pre-determined level of accuracy is reached.

4. The system of claim 1 , wherein the calibration module is further configured to determine when the calibration data does not satisfy a pre-determined level of accuracy.

5. The system of claim 4 , wherein the calibration module is further configured to, upon determining that the calibration data does not satisfy the pre-determined level of accuracy:
select additional images, from the plurality of images, that depict the target that presents the fiducial feature;
determine, in each of the selected additional images, an observed location of the fiducial feature in the respective selected additional image; and
generate calibration data based at least in part on the observed locations of the fiducial feature in the selected additional images.

6. The system of claim 1 , further comprising:
an image projector configured to project the fiducial feature onto the target;
and wherein the calibration data is for calibrating the image projector.

7. The system of claim 1 , wherein the fiducial feature is part of, or affixed to, the target.

8. The system of claim 1 , wherein the calibration matrix metadata is based at least in part on at least one of intrinsic parameters, extrinsic parameters, a rolling convergence accuracy value, a date, or a time.

9. The system of claim 1 , wherein the calibration matrix metadata is based at least in part on a focal point, a principal point, or both a focal point and a principal point.

10. The system of claim 1 , wherein the camera is configured to move from a first position to a second position during at least a portion of acquisition of the video.

11. The system of claim 1 , wherein the calibration data is used to adjust or set one or more extrinsic or intrinsic parameters of the camera.

12. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:
receiving video comprising multiple images that depict a target that presents a fiducial feature, the target residing within a first portion of an environment;
receiving video comprising multiple images of a second portion of the environment that is different than the first portion, wherein the second portion of the environment does not include the target that presents the fiducial feature;
identifying the multiple images that depict the target that presents the fiducial feature;
determining, in each of the multiple images that depict the target that presents the fiducial feature, an observed location of the fiducial feature in the respective image;
generating calibration data based at least in part on the observed locations of the fiducial feature in the multiple images that depict the fiducial feature;
generating calibration matrix metadata based at least in part on the calibration data; and
associating at least a portion of the calibration matrix metadata with the video.

13. The one or more non-transitory computer readable storage of claim 12 , wherein the calibration matrix metadata describes a calibration of a camera that acquired the video.

14. The one or more non-transitory computer readable storage of claim 12 , wherein the calibration matrix metadata describes a calibration of an image projector that projected the fiducial feature.

15. The one or more non-transitory computer readable storage of claim 12 , wherein the multiple images of the video comprise at least one image acquired using a first field-of-view and at least one other image acquired using a second, distinct field-of-view.

16. The one or more non-transitory computer readable storage of claim 12 , wherein the calibration matrix metadata comprises one or more intrinsic parameters.

17. The one or more non-transitory computer readable storage of claim 12 , wherein the calibration matrix metadata comprises one or more extrinsic parameters.

18. One or more non-transitory computer-readable storage media storing instructions that, when executed by one or more processors, cause the one or more processors to perform acts comprising:
accessing video comprising a first set of images, wherein each image of the first set of images depicts a target that includes one or more fiducial features, the target residing in a first portion of an environment, the first set of images being associated with calibration matrix metadata;
accessing video comprising a second set of images of the environment, wherein each image of the second set of images does not depict the target that includes the one or more fiducial features;
generating calibrated video by applying the calibration matrix metadata of the first set of images to the second set of images; and
outputting the calibrated video.

19. The one or more non-transitory computer readable storage of claim 18 , wherein the calibration matrix metadata is based at least in part on calibration data generated from observed locations of one or more fiducial features in images of the first set of images.

20. The one or more non-transitory computer readable storage of claim 18 , wherein the calibration matrix metadata comprises one or more of the following intrinsic parameters:
a focal point,
a principal point,
a skew coefficient, or
a distortion coefficient.

Assignees

• Crump Edward Dietz
• Puri Manika
• Amazon Tech Inc

Inventors

• Crump Edward Dietz
• Puri Manika

Classifications

• H04N17/02 Primary

  for colour television signals · CPC title

• H04N9/3194 Primary

  including sensor feedback · CPC title

• H04N17/002

  for television cameras · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title