Identifying a position of a marker in an environment

Patent metadata

Field	Value
Publication number	US-10489651-B2
Application number	US-201715607496-A
Country	US
Kind code	B2
Filing date	May 28, 2017
Priority date	Apr 14, 2017
Publication date	Nov 26, 2019
Grant date	Nov 26, 2019

Abstract

A technique is described herein for determining the position of at least one previously-placed physical marker in a physical environment. In one approach, the technique detects the marker at plural vantage points in the environment, to yield plural instances of marker information. The technique then computes the position of the marker based on the plural instances of marker information collected at the plural vantage points. The technique may also provide a movement indicator that assists the user in moving to specified vantage points in the physical environment. The technique may use the identified position(s) of the marker(s) to accurately place virtual objects relative to real-world objects in a modified-reality world.

First claim

What is claimed is:

1. A method performed by one or more computing devices, the method comprising:
scanning image information reflecting different regions of environment for an indication of a physical marker, the image information being captured by at least one image capture device;
in an instance when an individual region being scanned does not include the physical marker, outputting a search indicator conveying that the environment is being scanned for the physical marker;
responsive to detecting an indication that the physical marker is present in a particular region of the environment, initiating a position-determining process for the physical marker, the position-determining process comprising:
receiving a first instance of image information captured by the at least one image capture device while positioned at a first vantage point in the environment;
identifying a first instance of marker information in the first instance of image information that depicts the physical marker;
instructing, via a user interface, a user to move to a second vantage point within the environment relative to the physical marker;
receiving a second instance of image information captured by said at least one image capture device while positioned at the second vantage point in the environment;
processing the second instance of image information to identify a second instance of marker information that depicts the physical marker; and
determining a position of the physical marker in the environment relative to the at least one image capture device based on at least the first instance of marker information and the second instance of marker information.

2. The method of claim 1 , wherein said one or more computing devices are associated with a head-mounted display.

3. The method of claim 1 , wherein the search indicator comprises a display feature presented on the user interface.

4. The method of claim 1 , further comprising:
after detecting the indication that the physical marker is present in the particular region of the environment, displaying a progress indicator that conveys an extent of progress of said one or more computing devices in confirming that the first instance of marker information depicts the physical marker.

5. The method of claim 4 , wherein the progress indicator corresponds to a display feature that grows in length in proportion to the progress.

6. The method of claim 4 ,
further comprising displaying a representation of the first instance of marker information on a display device,
wherein the progress indicator corresponds to a display feature that advances around the representation of the first instance of marker information in proportion to the progress.

7. The method of claim 1 , further comprising:
providing, on the user interface, a movement indicator that describes a plurality of vantage points to which the user is invited to move along a path.

8. The method of claim 7 , further comprising:
as individual instances of marker information are successfully identified at individual vantage points, updating the movement indicator to indicate that the individual instances of marker information have been successfully identified at the individual vantage points.

9. The method of claim 1 , further comprising receiving at least one additional instance of image information, and, for each additional instance of image information that is collected at a particular vantage point, identifying at least one additional instance of marker information.

10. The method of claim 1 , wherein said determining comprises:
using the first instance of marker information to virtually project a first ray into the environment based on at least a position of the first instance of marker information in the first instance of image information and a placement of said at least one image capture device in the environment while at the first vantage point;
using the second instance of marker information to virtually project a second ray into the environment based on at least a position of the second instance of marker information in the second instance of image information and a placement of said at least one image capture device in the environment while at the second vantage point;
identifying an intersection of the first ray and the second ray; and
determining the position of the physical marker based at least on the intersection.

11. The method of claim 10 , further comprising constraining the position of the physical marker to lie on an identified reconstructed surface, corresponding to a detected physical surface in the environment.

12. The method of claim 1 , further comprising repeating the method to identify another position of at least one additional physical marker that is present in the environment.

13. The method of claim 1 , further comprising placing a virtual object in a modified-reality world in relation to the position of the physical marker, and presenting the virtual object on a display device.

14. One or more computing devices comprising:
a hardware processor; and
a storage resource storing machine-readable instructions which, when executed by the hardware processor, cause the hardware processor to:
using at least one image capture device, scan an environment for an initial indication that a real-world marker is present in a target region;
prior to identifying the initial indication that the real-world marker is present in the target region, output a search indicator conveying that the one or more computing devices are scanning for the real-world marker;
after identifying the initial indication that the real-world marker is present in the target region, successively receive plural instances of image information captured at plural vantage points in the environment by the at least one image capture device;
process the plural instances of image information to identify instances of marker information in the respective instances of image information, the instances of marker information depicting the real-world marker that is present in the environment; and
determine a position of the real-world marker in the environment relative to the at least one image capture device based at least on the instances of marker information.

15. The one or more computing devices of claim 14 , wherein the machine-readable instructions, when executed by the hardware processor, cause the hardware processor to:
detect the initial indication that the real-world marker is present in the target region by identifying a group of pixels having a particular color associated with the real-world marker.

16. The one or more computing devices of claim 14 , wherein the machine-readable instructions, when executed by the hardware processor, cause the hardware processor to determine the position of the real-world marker by:
virtually projecting a plurality of rays into the environment, each ray being based on at least: a position of an instance of marker information in a corresponding instance of image information and a placement of said at least one image capture device in the environment while at a corresponding vantage point in the environment;
identifying intersection information that describes a manner in which the plurality of rays intersect; and
determining the position of the real-world marker based at least on the intersection information.

17. The one or more computing devices of claim 16 , wherein the machine-readable instructions, when executed by the hardware processor, cause the hardware processor to:
constrain the position of the real-world marker to lie on an identified reconstructed

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Luccin Karim Audrey
• Ghaly Marcus Philip
• Paolantonio Sergio
• Bassir Brian Zhian
• Felice Michael Jordon

Classifications

• G06V20/20

  in augmented reality scenes · CPC title

• G06T7/73 Primary

  using feature-based methods · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• G06T2200/24

  involving graphical user interfaces [GUIs] · CPC title