3D mapping with flexible camera rig

Patent metadata

Field	Value
Publication number	US-9759918-B2
Application number	US-201414267714-A
Country	US
Kind code	B2
Filing date	May 1, 2014
Priority date	May 1, 2014
Publication date	Sep 12, 2017
Grant date	Sep 12, 2017

Abstract

Embodiments related to mapping an environment of a machine-vision system are disclosed. For example, one disclosed method includes acquiring image data resolving one or more reference features of an environment and computing a parameter value based on the image data, wherein the parameter value is responsive to physical deformation of the machine-vision system.

First claim

The invention claimed is:

1. In a wearable machine-vision camera system subject to real-time physical deformation while worn, a method to map an environment, the method comprising:
acquiring a series of image frames resolving one or more reference features and one or more subject features of the environment;
locating the one or more reference features in the series of image frames;
computing, based on the reference features as located in the series of image frames, a parameter value responsive to a real-time physical deformation of the machine-vision camera system that reorients one or more cameras of the machine-vision camera system;
assigning coordinates to the one or more subject features based on the parameter value; and
presenting on a display a virtual display image responsive to the coordinates of the one or more subject features.

2. The method of claim 1 , wherein the machine-vision camera system is a wearable system, the method further comprising assigning coordinates to one or more of the machine-vision camera system and a wearer thereof based on the parameter value.

3. The method of claim 1 , wherein the machine-vision camera system is coupled to a near-eye display projector, the method further comprising assigning coordinates to a virtual display frame of the near-eye display projector based on the parameter value.

4. The method of claim 1 , wherein the parameter value is one of a plurality of parameter values responsive to the real-time physical deformation, and wherein the plurality of parameter values are computed based on the series of image frames and are used to assign coordinates to the one or more subject features.

5. The method of claim 4 , wherein the parameter value is responsive to a twist of the machine-vision camera system about an axis of the machine-vision camera system.

6. The method of claim 4 , wherein the parameter value is responsive to a bend of the machine-vision camera system about a fixed point on the machine-vision camera system.

7. The method of claim 1 , wherein locating the one or more reference features includes locating the one or more reference features in concurrent image frames acquired by the machine-vision camera system.

8. The method of claim 1 , wherein locating the one or more reference features includes locating the one or more reference features in image frames acquired at different times by the machine-vision camera system.

9. The method of claim 1 , further comprising receiving output from an inertial measurement unit in the machine-vision camera system to determine an orientation of the machine-vision camera system.

10. A wearable machine-vision system subject to real-time physical deformation while worn, the machine-vision system comprising:
one or more cameras configured to acquire a series of image frames resolving one or more reference features and one or more subject features of an environment;
a binocular near-eye display projector;
a flexible, wearable rig to which the one or more cameras and near-eye display projector are fixed; and
a logic machine operatively coupled to an instruction-storage machine and configured to locate the one or more reference features in the series of image frames, compute, based on the reference features as located in the series of image frames, a parameter value responsive to a real-time physical deformation of the machine-vision system that reorients the one or more cameras of the machine-vision system, assign coordinates to the one or more subject features based on the parameter value, and present on the near-eye display projector a virtual display image responsive to the coordinates of the one or more subject features.

11. The machine-vision system of claim 10 , wherein the one or more cameras include cameras with overlapping fields of view.

12. The machine-vision system of claim 10 , wherein the one or more cameras include cameras oriented for stereoscopic depth imaging.

13. The machine-vision system of claim 10 , wherein the one or more cameras include a camera configured for time-of-flight depth sensing.

14. The machine-vision system of claim 10 , further comprising an inertial measurement unit responsive to an orientation of the machine-vision system and/or a change in position of the machine-vision system.

15. In a head-wearable machine-vision camera system subject to real-time physical deformation while worn, a method to map an environment, the method comprising:
acquiring a series of image frames resolving one or more reference features and one or more subject features of the environment, the image frames being acquired by one or more cameras arranged on a flexible rig;
locating the one or more reference features in the series of image frames;
computing, based on the reference features as located in the series of image frames, a plurality of parameter values, each responsive to an extent of a real-time physical deformation of the rig in a corresponding deformation mode, the physical deformation being such as to reorient the one or more cameras of the machine-vision camera system;
assigning coordinates to the one or more subject features based on each of the parameter values, wherein the assigned coordinates are invariant to the extent of the physical deformation;
assigning coordinates to a display componentry arranged on the flexible rig; and
presenting a virtual display image on the display componentry responsive to the coordinates of the one or more subject features and further responsive to the coordinates assigned to the display componentry.

16. The method of claim 15 , wherein computing the plurality of parameter values includes computing at regular intervals during acquisition of the series of image frames.

17. The method of claim 15 , wherein the parameter value is a value of a parameter in a deformation model applied to the rig, the parameter representing one or more of a twist and a bend of the rig.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Ebstyne Michael John
• Schaffalitzky Frederik
• Steedly Drew
• Klein Georg
• Eade Ethan
• Grabner Michael

Classifications

• G06T7/85

  Stereo camera calibration · CPC title

• G06T2207/30204

  Marker · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G06T7/80

  Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title