Systems and methods for camera calibration with a fiducial of unknown position on an articulated arm of a programmable motion device

Patent metadata

Field	Value
Publication number	US-12415280-B2
Application number	US-202318377086-A
Country	US
Kind code	B2
Filing date	Oct 5, 2023
Priority date	Feb 6, 2020
Publication date	Sep 16, 2025
Grant date	Sep 16, 2025

Abstract

A system is disclosed for providing extrinsic calibration of a camera to a relative working environment of a programmable motion device that includes an end-effector. The system includes a fiducial located at or near the end-effector, at least one camera system for viewing the fiducial as the programmable motion device moves in at least three degrees of freedom, and for capturing a plurality of images containing the fiducial, and a calibration system for analyzing the plurality of images to determine a fiducial location with respect to the camera to permit calibration of the camera with the programmable motion device.

First claim

What is claimed is:

1. A calibration unit comprising:
a coupling portion for removably coupling to a programmable motion device;
a transverse portion with a planar shape that extends transverse to the coupling portion, said transverse portion including a top side and a bottom side thereof;
a plurality of light emitting diodes (LEDs), wherein the plurality of LEDs are disposed on the top side and the bottom side of the transverse portion; and
control circuitry in communication with the plurality of LEDs and in communication with a calibration processing system for analyzing images of the calibration unit including at least one of the plurality of LEDs from a camera projection point of a camera, and for providing calibration information regarding the programmable motion device and the camera projection point.

2. The calibration unit of claim 1 , wherein the plurality of LEDs provides intermittent visible illumination.

3. The calibration unit of claim 1 , wherein the plurality of LEDs provides intermittent infra-red illumination.

4. The calibration unit of claim 1 , wherein the calibration unit further includes a retroreflective fiducial.

5. The calibration unit of claim 4 , wherein the retroreflective fiducial is at least partially spherically shaped.

6. The calibration unit of claim 1 , wherein the calibration processing system constructs a matrix for each of the plurality of images.

7. The calibration unit of claim 6 , wherein the calibration processing system concatenates extrinsic parameters of the camera to the programmable motion device.

8. The calibration unit of claim 7 , wherein the calibration processing system further employs non-linear least square optimization.

9. The calibration unit of claim 1 , wherein the camera is positioned on an automated carrier.

10. The calibration unit of claim 1 , wherein the plurality of LEDs includes a first pair of LEDs that are positioned on mutually opposing sides of the transverse portion.

11. The calibration unit of claim 10 , wherein the plurality of LEDs further includes a second pair of LEDs that are positioned on mutually opposing sides of the transverse portion.

12. The calibration unit of claim 11 , wherein the first pair of LEDs are visible LEDs and the second pair of LEDs are infra-red LEDs.

13. A calibration unit comprising:
a coupling portion for removably coupling to a programmable motion device;
a transverse portion with a planar shape that extends transverse to the coupling portion, the transverse portion including a top side and a bottom side thereof;
a plurality of light emitting diodes (LEDs), wherein a first pair of the plurality of LEDs are disposed on the top side and a bottom side of the transverse portion and a second pair of the plurality of LEDs are on the bottom side of the transverse portion, wherein the first pair of the plurality of LEDs are visible LEDs and the second pair of the plurality of LEDs are infra-red LEDs; and
control circuitry in communication with the plurality of LEDs and in communication with a calibration processing system for analyzing data with respect to a camera projection point of a camera regarding the plurality of LEDs, and for providing calibration information regarding the programmable motion device and the camera projection point.

14. The calibration unit of claim 13 , wherein the first pair of the plurality of LEDs provides intermittent visible illumination.

15. The calibration unit of claim 13 , wherein the second pair of the plurality of LEDs provides intermittent infra-red illumination.

16. The calibration unit of claim 13 , wherein the calibration unit further includes a retroreflective fiducial.

17. The calibration unit of claim 16 , wherein the retroreflective fiducial is at least partially spherically shaped.

18. The calibration unit of claim 13 , wherein the calibration processing system concatenates extrinsic parameters of the camera projection point to the programmable motion device.

19. The calibration unit of claim 18 , wherein the calibration processing system employs non-linear least square optimization.

20. A method of providing calibration of a camera to a relative working environment of a programmable motion device that includes an end-effector, said method comprising:
attaching, using the programmable motion device, a calibration unit from a rack to the end effector, the calibration unit including a coupling portion for removably coupling the calibration unit to the end effector, a transverse portion with a planar shape that extends transverse to the coupling portion, and a plurality of light emitting diodes (LEDs), at least one of the plurality of LEDs being on a top side of the transverse portion, and at least another of the plurality of LEDs being on a bottom side of the transverse portion;
capturing a plurality of images of the calibration unit on the end-effector from a camera projection point of a camera;
analyzing the plurality of images that include one or more of the plurality of LEDs on the calibration unit; and
providing calibration information regarding the camera projection point and the programmable motion device.

21. The method of claim 20 , wherein at least one of the plurality of LEDs provides intermittent visible illumination.

22. The method of claim 20 , wherein at least one of the plurality of LEDs provides intermittent infra-red illumination.

23. The method of claim 20 , wherein the calibration unit further includes a retroreflective fiducial.

24. The method of claim 23 , wherein the retroreflective fiducial is at least partially spherically shaped.

25. The method of claim 23 , wherein analyzing the plurality of images includes constructing a matrix for each of the plurality of images of the fiducial.

26. The method of claim 25 , wherein analyzing the plurality of images further includes concatenating extrinsic parameters of the camera to the programmable motion device.

27. The method of claim 26 , wherein analyzing the plurality of images further includes employing non-linear least square optimization.

28. The method of claim 20 , wherein the plurality of LEDs includes a first pair of LEDs that are visible LEDs and a second pair of LEDs that are infra-red LEDs.

Assignees

• Berkshire Grey Operating Company Inc

Inventors

• Ding Yu
• GEYER Christopher
• Galup Luis
• Kittredge Jeffrey
• Koletschka Thomas
• Kumar Lakshman
• Nellithimaru Anjana
• O'Hern Ryan
• Saslaw Jeremy

Classifications

• G05B2219/37009

  Calibration of vision system, camera, adapt light level · CPC title

• G06V10/147

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06T2207/30208

  Marker matrix · CPC title

• B25J9/1692

  Calibration of manipulator · CPC title