Attribute detection tools for mixed reality

Patent metadata

Field	Value
Publication number	US-10325407-B2
Application number	US-201615395513-A
Country	US
Kind code	B2
Filing date	Dec 30, 2016
Priority date	Sep 15, 2016
Publication date	Jun 18, 2019
Grant date	Jun 18, 2019

Abstract

Techniques described herein include mixed reality tools, referred to as HoloPaint, that allow use of any of a variety of sensors to determine physical parameters of real objects in a mixed reality environment. HoloPaint may correlate current measurements of the real world with past measurements to perform inventory management, analysis of changes of physical parameters of real objects and environments, and so on. A user may select which parameter to analyze by selecting a particular type of virtual paint, such as for drawing onto an object to be analyzed.

First claim

What is claimed:

1. A method comprising:
receiving a first set of values for a physical attribute of an object measured at a first time;
receiving instructions to virtually paint a portion of the object in a mixed reality environment;
measuring a second set of values for the physical attribute of the portion of the object at a second time in the mixed reality environment;
comparing the second set of values to the first set of values to produce a comparison;
determining a type of paint to apply onto the object, wherein the determining is based, at least in part, on the instructions and the comparison; and
painting the object such that the paint reflects a change for the physical attribute from a first time to a second time.

2. The method as claim 1 recites, wherein the type of paint is based at least in part on color, brightness, albedo, or hue.

3. The method as claim 1 recites, further comprising: determining a placement location of the paint, wherein the determining is based, at least in part, on the instructions.

4. The method as claim 1 recites, wherein the instructions comprise drawing instructions that designate the portion of the object to be painted.

5. The method as claim 1 recites, wherein the object is a real object.

6. The method as claim 1 recites, wherein the physical attribute is a kinematic parameter.

7. The method as claim 1 recites, wherein the first set of values and the second set of values for the physical attribute comprise respective mappings of the physical attribute over at least a portion of the object.

8. A method comprising:
receiving stored inventory data for a set of objects in a first state; virtually painting at least a portion of the set of objects in a second state in a mixed reality environment;
determining current inventory data for the virtually painted at least the portion of the set of objects in the second state;
comparing the current inventory data to the stored inventory data to produce a comparison; and
determining a change in inventory for the set of objects in the second state relative to the set of objects in the first state, wherein the determining is based, at least in part, on the comparison.

9. The method as claim 8 recites, wherein the set of objects in the first state comprises a first number of the objects and the set of objects in the second state comprises a second number of the objects, wherein the second number is different from the first number.

10. The method as claim 8 recites, wherein the stored inventory data and the current inventory data comprise a location and number of individual objects of the set of objects in the first state and the second state, respectively.

11. The method as claim 8 recites, wherein virtually painting at least a portion of the set of objects in the second state comprises drawing a virtual line across the at least the portion of the set of objects in the second state.

12. The method as claim 8 recites, further comprising:
causing virtual paint to be applied to portions of the set of objects in the second state that are different from corresponding portions of the set of objects in the first state to distinguish from portions of the set of objects in the second state that are same as corresponding portions of the set of objects in the first state.

13. The method as claim 8 recites, further comprising:
causing virtual paint to be applied to portions of the set of objects in the second state that are same as corresponding portions of the set of objects in the first state to distinguish from portions of the set of objects in the second state that are different from corresponding portions of the set of objects in the first state.

14. The method as claim 8 recites, wherein the stored inventory data comprises an image captured at a time that the set of objects is in the first state.

15. The method as claim 8 recites, wherein the set of objects are real objects.

16. The method as claim 8 recites, wherein the comparing the current inventory data to the stored inventory data comprises comparing a subset of the current inventory data to the stored inventory data, wherein the subset is based, at least in part, on at least one physical attribute of the set of objects in the second state.

17. A system comprising:
a mixed reality display device operable in a mixed reality environment; and a device communicatively coupled to the mixed reality display device, the device comprising: one or more processors;
memory; and
one or more modules stored in the memory and executable by the one or more processors to perform operations comprising:
determining differences between a first map of a physical parameter of an object and a second map of the physical parameter of the object; and
virtually painting a particular region of the object, wherein the location of the particular region is based, at least in part, on the determined differences.

18. The system as claim 17 recites, wherein the first map of the physical parameter of the object comprises first values of the physical parameter at a first time and the second map of the physical parameter of the object comprises second values of the physical parameter at a second time.

19. The system as claim 17 recites, wherein the physical parameter is a kinematic attribute of the object.

20. The system as claim 17 recites, wherein the physical attribute is a compound physical attribute comprising two or more parameters of the object.

Assignees

• Microsoft Technology Licensing Llc

Inventors

• Lanier Jaron
• Amores Fernandez Judith

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T19/20

  Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06T11/001

  Physics · mapped topic