Calibration and transformation of a camera system's coordinate system

Patent metadata

Field	Value
Publication number	US-9542743-B2
Application number	US-201514881457-A
Country	US
Kind code	B2
Filing date	Oct 13, 2015
Priority date	Jul 8, 2011
Publication date	Jan 10, 2017
Grant date	Jan 10, 2017

Abstract

Systems and methods are disclosed that determine a mapping between a first camera system's coordinate system and a second camera system's coordinate system; or determine a transformation between a robot's coordinate system and a camera system's coordinate system, and/or locate, in a robot's coordinate system, a tool extending from an arm of the robot based on the tool location in the camera's coordinate system. The disclosed systems and methods may use transformations derived from coordinates of features found in one or more images. The transformations may be used to interrelate various coordinate systems, facilitating calibration of camera systems, including in robotic systems, such as an image-guided robotic systems for hair harvesting and/or implantation.

First claim

The invention claimed is:

1. A method of locating, in a robot's coordinate system, a tool extending from an arm of the robot, the method comprising:
determining from an image data a location of the tool in a camera system's coordinate system, wherein determining comprises detecting from the image data first and second peripheral sidewall edge segments corresponding to sides of the tool within a field of view of the camera system, and computing a centerline based on an average between the detected first and second peripheral sidewall edge segments; and
translating the location of the tool from the camera system's coordinate system into the robot's coordinate system using a predetermined transformation.

2. The method according to claim 1 , further comprising locating in the camera system's coordinate system a position of a distal end of the tool along the centerline; and wherein translating the location of the tool comprises translating the distal end of the tool.

3. The method according to claim 1 , wherein the centerline bisects an angle formed between the first and second peripheral sidewall edge segments.

4. The method according to claim 1 , wherein the determining step comprises locating first and second data transitions in individual rows of the image data to establish a first and second set of data transitions, calculating the first and second peripheral sidewall edge segments by performing a regression from the respective first and second sets of data transitions, and wherein the locating step comprises locating a distal-end data transition in the image data along the centerline such that the location of the distal-end data transition represents the spatial position of the distal end in the camera system's coordinate system.

5. The method according to claim 4 , wherein the first and second data transitions comprise one or more of contrast, intensity, color, or brightness transitions of the image data.

6. The method according to claim 4 , wherein the determining step further comprises processing the image data such that the distal end is represented as a unique value within the image data.

7. The method according to claim 4 , wherein the determining step further comprises binarizing the image data such that the distal end is represented by a first unique color and background features are represented by a second unique color.

8. The method according to claim 4 , wherein the data transitions comprise grayscale edge points.

9. The method according to claim 4 , wherein the regression comprises a best-fit algorithm that approximately connects respective first and second sets of data transitions and/or a least-squares fit that approximately connects respective first and second sets of data transitions.

10. The method according to claim 1 , wherein the tool comprises a follicular unit harvesting or implantation tool, the method further comprising moving the tool adjacent a follicular unit or an implantation site for hair harvesting or implantation.

11. The method according to claim 1 , further comprising: capturing image data, wherein the image data comprises two-dimensional data from a first and a second camera system which capture the image data at substantially the same time, and calculating a three-dimensional location from the two-dimensional image data.

12. The method according to claim 11 , further comprising identifying relative positions and magnification of the first and second camera system, and wherein a set of two or more distinct calibration features are used for camera identification.

13. The method according to claim 12 , wherein identifying the magnification comprises differentiating between left and right side cameras.

14. The method according to claim 1 , further comprising determining an offset between a flange of the tool and a tip of the tool to automatically compute the tool tip location in three-dimensions.

15. The method of claim 1 , further comprising capturing an image of a feature of interest relative to the tool and generating instructions that cause a controller to move the arm of the robot to a desired position and orientation relative to the feature of interest.

16. A system for locating, in a robot's coordinate system, a tool extending from an arm of the robot, the system comprising:
at least one non-transitory storage medium storing instructions and one or more modules for executing operations on image data, the one or more modules comprising instructions for:
determining from image data a location of a tool in a camera system's coordinate system, wherein determining comprises detecting from the image data first and second peripheral sidewall edge segments corresponding to sides of the tool within a field of view of the camera system, and computing a centerline based on an average between the detected first and second peripheral sidewall edge segments; and
translating the location of the tool from the camera system's coordinate system into the robot's coordinate system using a predetermined transformation.

17. The system of claim 16 , further comprising a camera system having a coordinate system and configured to capture the image data.

18. The system of claim 17 , wherein the camera is substantially fixed to a tool assembly comprising the tool.

19. The system of claim 17 , wherein the camera system comprises two cameras which are not parallel to each other.

20. The system of claim 17 , further comprising a means for changing focus of the camera to focus on the distal end of the tool.

21. The system of claim 16 , further comprising lighting disposed on the tool.

22. The system of claim 16 , wherein the tool comprises a hair harvesting tool or a hair implanting tool.

23. The system of claim 16 , further comprising instructions to detect the first and second peripheral sidewall edge segments of the tool by locating first and second data transitions in individual rows of the image data to establish a first and second set of data transitions.

24. The system of claim 23 , further comprising instructions to preprocess the image data to make the data transitions distinct.

25. The system of claim 23 , further comprising instructions to detect the first and second peripheral sidewall edge segments of the tool by excluding from the first and second sets of data transitions, a first or second data transition having a distance from the respective first and second initial peripheral sidewall edge segments that exceeds a predetermined threshold.

26. The system of claim 25 , further comprising instructions to re-calculate the first and second peripheral sidewall edge segments by performing a regression from the respective first and second sets of data transitions.

Assignees

• Restoration Robotics Inc

Inventors

• Tenney John A
• Burd Erik R
• Zhang Hui
• Biro Robert F

Classifications

• G05B2219/39022

  Transform between measuring and manipulator coordinate system · CPC title

• G05B19/4086

  Coordinate conversions; Other special calculations · CPC title

• G06T7/0042

  Physics · mapped topic

• G06T2207/10012

  Stereo images · CPC title

• B25J9/1697

  Vision controlled systems · CPC title