Systems and methods of object shape and position determination in three-dimensional (3D) space

Patent metadata

Field	Value
Publication number	US-9697643-B2
Application number	US-201514710499-A
Country	US
Kind code	B2
Filing date	May 12, 2015
Priority date	Jan 17, 2012
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

Methods and systems for capturing motion and/or determining the shapes and positions of one or more objects in 3D space utilize cross-sections thereof. In various embodiments, images of the cross-sections are captured using a camera based on edge points thereof.

First claim

What is claimed is:

1. A system of identifying a position and a shape of a portion of an object in a three-dimensional (3D) space, the system comprising:
one or more processors coupled to

a memory, the memory loaded with computer instructions that, when executed on the one or more processors, implement a method including:
capturing, with a plurality of cameras, each camera of the plurality of cameras having a particular vantage point, two or more images generated by casting an output from at least one source onto the portion of the object;
analyzing the two or more images captured by the cameras from the particular vantage points to computationally represent the portion of the object, as captured, as one or more mathematically represented 3D surfaces, each 3D surface corresponding to a cross-section of the portion of the object, based at least in part on a plurality of edge points of the portion of the object in the image, tangent lines extending from the plurality of cameras to at least two edge points of the plurality of edge points and a centerline corresponding to the tangent lines; and
reconstructing the position of and the shape fitting at least the portion of the object in the 3D space based at least in part on the plurality of edge points and the centerline.

2. The system of claim 1 , wherein at least a portion of at least one of the 3D surfaces is determined from a correlation of slices that represent all or part of the object.

3. The system of claim 2 , wherein at least one of the slices is tilted with respect to one or more other of the slices.

4. The system of claim 2 , wherein the correlating comprises defining a 3D surface that best fits the slices.

5. The system of claim 1 , wherein:
the analyzing comprises determining the centerline by identifying diagonal line segments that connect opposite corners of an intersection region formed from an intersection of two pairs of intersecting tangent lines selected from the tangent lines, identifying midpoints of the diagonal line segments, and identifying a line segment joining the midpoints as the centerline; and
the reconstructing comprises fitting at least one of the 3D surfaces to a point at which at least one of the tangent lines is substantially tangent to the object, as represented, and positioned about the centerline.

6. The system of claim 5 , wherein the reconstructing further comprises stacking two or more of the 3D surfaces.

7. The system of claim 1 , wherein at least one of the 3D surfaces approximates a contour of the cross-section of the portion of the object.

8. The system of claim 1 , wherein at least one of the 3D surfaces is symmetrical in at least one plane.

9. The system of claim 8 , wherein at least one of the 3D surfaces is selected from a group consisting of ellipsoids, hyperboloids, cylinders and other quadratic surfaces.

10. The system of claim 1 , wherein the analyzing comprises revising characteristic parameters of at least one of the 3D surfaces based upon results of the analyzing as conducted on images obtained at a different time.

11. The system of claim 1 , wherein:
at least one of the 3D surfaces is defined according to characteristic parameters defining characteristics of the at least one of the 3D surfaces; and
the analyzing comprises assuming a characteristic parameter of the at least one of the 3D surfaces.

12. The system of claim 11 , wherein the assuming is conducted according to a type of the object.

13. The system of claim 1 , wherein:
at least one of the 3D surfaces is defined according to characteristic parameters defining characteristics of the at least one of the 3D surfaces; and
the analyzing comprises assuming a typical characteristic parameter of the at least one of the 3D surfaces.

14. The system of claim 1 , wherein the object comprises multiple members.

15. The system of claim 14 , wherein one or more of the multiple members is separately articulating.

16. The system of claim 15 , wherein spatial relationships between multiple members change as a subject comprising the multiple members moves.

17. The system of claim 1 , wherein the portion of the object is selected from a group consisting essentially of fingers, finger segments, thumb segments, hands, palms, arms, legs, a core of a person and a trunk of a person.

18. The system of claim 1 , wherein the analyzing comprises analyzing additional images and determining an object motion of the object.

19. The system of claim 18 , wherein the object motion determines a pose.

20. The system of claim 18 , wherein the determining of the object motion of the object comprises determining a movement of digits indicated by a sequence of models captured across time.

21. The system of claim 20 , wherein the object is any type of object.

22. The system of claim 20 , wherein the object is a hand and wherein motion of the hand as captured is used to control equipment.

23. The system of claim 1 , wherein the capturing comprises capturing images of a viewed portion of the object.

24. A non-transitory computer readable medium storing a plurality of instructions for programming one or more processors to identify a position and a shape of a portion of an object in a three-dimensional (3D) space, the instructions, when executed on the one or more processors, implementing a method including:
capturing, with a plurality of cameras, each camera of the plurality of cameras having a particular vantage point, two or more images generated by casting an output from at least one source onto the portion of the object;
analyzing the two or more images captured by the cameras from the particular vantage points to computationally represent the portion of the object, as captured, as one or more mathematically represented 3D surfaces, each 3D surface corresponding to a cross-section of the portion of the object, based at least in part on a plurality of edge points of the portion of the object in the image, tangent lines extending from the plurality of cameras to at least two edge points of the plurality of edge points and a centerline corresponding to the tangent lines; and
reconstructing the position of and the shape fitting at least the portion of the object in the 3D space based at least in part on the plurality of edge points and the centerline.

Assignees

• Leap Motion Inc

Inventors

• Holz David S

Classifications

• G06T17/00 Primary

  Three-dimensional [3D] modelling for computer graphics · CPC title

• G06V10/25 Primary

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06T7/75

  involving models · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title