Systems and methods of locating a control object appendage in three dimensional (3D) space

Patent metadata

Field	Value
Publication number	US-9945660-B2
Application number	US-201514723370-A
Country	US
Kind code	B2
Filing date	May 27, 2015
Priority date	Jan 17, 2012
Publication date	Apr 17, 2018
Grant date	Apr 17, 2018

Abstract

Methods and systems for capturing motion and/or determining the shapes and positions of one or more objects in 3D space utilize cross-sections thereof. In various embodiments, images of the cross-sections are captured using a camera based on reflections therefrom or shadows cast thereby.

First claim

What is claimed is:

1. A system of locating a control object appendage in three dimensional (3D) space, the system including:
one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:
recording images of a control object appendage in 3D space using at least two geometrically distinct predetermined vantages;
calculating by a processor four co planar tangents to observed edges of the control object appendage from the recorded images;
fitting an ellipse by a processor to a cross-section of the control object appendage by selecting the ellipse from a family of ellipses that fit the four co planar tangents using an assumed parameter;
repeatedly fitting contiguous cross-sections to the control object appendage; and
constructing a model of the control object appendage in 3D space from the contiguous cross-sections.

2. The system of claim 1 , further configured to substitute one or more fitted parameters of a first fitted ellipse for one or more of the four co planar tangents and the assumed parameter when fitting an adjacent second ellipse.

3. The system of claim 1 , further configured to use a first fitted ellipse to filter fits of additional ellipses to the contiguous cross-sections.

4. The system of claim 2 , further including:
one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:
for a complex control object model that includes a palm and multiple fingers, applying the actions of claim 2 to construct multiple fingers of control object appendages; and
fitting cross sections of a palm to observed edges from the images in positions correlated with the multiple fingers.

5. The system of claim 1 , further including:
one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:
repeatedly applying the actions of claim 1 over time; and
calculating motion of the control object appendage over time based on differences between modeled locations of the control object appendage over time.

6. The system of claim 3 , further including:
one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:
repeatedly applying the actions of claim 3 over time; and
calculating motion of a complex control object over time based on differences between modeled locations of a complex control object over time.

7. The system of claim 1 , further configured to:
determine that only three co planar tangents are available in portions of the recorded images and fitting a circle instead of an ellipse for the portions of the recorded images, including:
calculating by a processor the three co planar tangents to observed edges of the control object appendage from the recorded images; and
fitting a circle by a processor to the cross-section of the control object appendage using at least the three co planar tangents.

8. The system of claim 7 , further including:
one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:
for a complex control object model that includes a palm and multiple fingers, applying the actions of claim 7 to construct multiple fingers of control object appendages; and
fitting cross sections of a palm to observed edges from the images in positions correlated with the multiple fingers.

9. The system of claim 7 , further including:
one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:
repeatedly applying the actions of claim 7 over time; and
calculating motion of the control object appendage over time based on differences between modeled locations of the control object appendage over time.

10. The system of claim 8 , further including:
one or more processors coupled to memory, the memory loaded with computer instructions that, when executed on the processors, implement actions including:
repeatedly applying a actions of claim 8 over time; and
calculating motion of a complex control object over time based on differences between modeled locations of the complex control object over time.

11. A non-transitory computer readable medium storing a plurality of instructions for programming one or more processors to locate a control object appendage in three dimensional (3D) space, the instructions, when executed on the processors, implementing actions including:
recording images of a control object appendage in 3D space using at least two geometrically distinct predetermined vantages;
calculating by a processor four co planar tangents to observed edges of the control object appendage from the recorded images;
fitting an ellipse by a processor to a cross-section of the control object appendage by selecting the ellipse from a family of ellipses that fit the four co planar tangents using an assumed parameter;
repeatedly fitting contiguous cross-sections to the control object appendage; and
constructing a model of the control object appendage in 3D space from the contiguous cross-sections.

12. The non-transitory computer readable medium of claim 11 , further configured to substitute one or more fitted parameters of a first fitted ellipse for one or more of the four co planar tangents and the assumed parameter when fitting an adjacent second ellipse.

13. The non-transitory computer readable medium of claim 11 , further configured to use a first fitted ellipse to filter fits of additional ellipses to the contiguous cross-sections.

14. The non-transitory computer readable medium of claim 12 , further including storing a plurality of instructions for programming one or more processors to locate a complex control object in 3D space, the instructions, when executed on the processors, implementing actions including:
for a complex control object model that includes a palm and multiple fingers, applying the actions of claim 12 to construct multiple fingers of control object appendages; and
fitting cross sections of a palm to observed edges from the images in positions correlated with the multiple fingers.

15. The non-transitory computer readable medium of claim 11 , further including storing a plurality of instructions for programming one or more processors to track motion of a control object appendage in 3D space, the instructions, when executed on the processors, implementing actions including:
repeatedly applying the actions of claim 11 over time; and
calculating motion of the control object appendage over time based on differences between modeled locations of the control object appendage over time.

16. The non-transitory computer readable medium of claim 13 , further including storing a plurality of instructions for programming one or more processors to locate a complex control object in 3D space, the instructions, when executed on the processors, implementing actions including:
repeatedly applying the actions of claim 13 over time; and
calculating motion of a complex control object over time based on differences between modeled locations of a complex control object over time.

17. The non-transitory computer readable medium of claim 11 , further configured to:
determine that only three co planar tangents are available in portions of the recorded images and fitting a circle

Assignees

• Leap Motion Inc

Inventors

• Holz David S

Classifications

• G06T2207/30196

  Human being; Person · CPC title

• G06T7/507

  from shading (G06T7/586 takes precedence) · CPC title

• G01B11/2433 Primary

  for measuring outlines by shadow casting · CPC title

• G06T7/292

  Multi-camera tracking · CPC title

• G06T7/593

  from stereo images · CPC title