Physical 3D renderer

Patent metadata

Field	Value
Publication number	US-9805506-B2
Application number	US-201414200950-A
Country	US
Kind code	B2
Filing date	Mar 7, 2014
Priority date	Mar 7, 2014
Publication date	Oct 31, 2017
Grant date	Oct 31, 2017

Abstract

The physical 3D renderer described herein renders one or more captured depth images as a physical 3D rendering. The physical 3D renderer can render physical 3D surfaces and structures in real time. In one embodiment the 3D renderer creates a physical three dimensional (3D) topological surface from captured images. To this end, a depth image of a surface or structure to be replicated is received (for example from a depth camera or depth sensor). Depth information is determined at a dense distribution of points corresponding to points in the depth image. In one embodiment the depth information corresponding to the depth image is fed to actuators on sliding shafts in an array. Each sliding shaft is adjusted to the depth in the depth image to create a physical 3D topological surface like the surface or structure to be replicated.

First claim

What is claimed is:

1. A computer-implemented process for creating a physical three dimensional (3D) topological surface from one or more images, comprising:
receiving a depth image of a surface to be replicated;
determining depth information at a dense distribution of points corresponding to points in the depth image so that the physical 3D topological surface that is replicated reflects the surface to be replicated;
feeding the depth information to actuators on sliding shafts in an array so that each sliding shaft is adjusted to the depth in the depth image to create the physical 3D topological surface that changes in real time as the surface to be replicated changes;
verifying the created physical 3D topological surface by comparing a depth image of the created physical 3D topological surface to a depth image of the surface to be replicated; and
correcting any discrepancies between the depth image of the surface to be replicated and the depth image of the created physical 3D topological surface by adjustments to the created physical 3D topological surface.

2. The computer-implemented process of claim 1 wherein the created 3D topological surface changes when a new depth image is received.

3. The computer-implemented process of claim 1 wherein adjustments to the created physical 3D topological surface are made by adjusting the sliding shafts.

4. The computer-implemented process of claim 1 wherein the created 3D topological surface is covered with a skin.

5. The computer-implemented process of claim 4 , further comprising projecting a colored image of the surface to be replicated onto the skin to colorize the created 3D topological surface.

6. The computer-implemented process of claim 1 wherein pneumatics are used to adjust the depths of the shafts.

7. The computer-implemented process of claim 1 wherein piezoelectric actuators are used to adjust the depths of the shafts.

8. The computer-implemented process of claim 1 wherein servos are used to adjust the depths of the shafts.

9. The computer-implemented process of claim 1 , wherein there are multiple actuators, each actuator controlling one shaft.

10. The computer-implemented process of claim 9 wherein the depth image of the created 3D topological surface captures the back of the created 3D topological surface.

11. The computer-implemented process of claim 1 , wherein the surface to be replicated is a physical surface.

12. The computer-implemented process of claim 1 wherein the spacing of the sliding shafts determine the resolution of the created 3D topological surface.

13. The computer-implemented process of claim 1 wherein the distal ends of the shafts further comprise an end cap.

14. The computer-implemented process of claim 1 wherein at least one shaft further comprises at least one light emitting source within the shaft to shine light from the distal end of the shaft.

15. The computer-implemented process of claim 14 wherein the light emitting source produces light of different colors.

16. The computer-implemented process of claim 15 wherein the created 3D topological surface can be colored using the light of different colors to replicate the colors of the surface to be replicated.

17. The computer-implemented process of claim 1 wherein more than one shaft can be controlled by an actuator.

18. A process for creating a physical 3D topological structure from images, comprising:
receiving a depth image of a structure to be replicated;
determining depth information at a dense distribution of points corresponding to points in the depth image so that the physical 3D topological structure that is replicated reflects the structure to be replicated;
feeding the depth information to an array of air jets that are covered with a skin;
adjusting pressurized air streams output by each air jet in the array based on the depth information to deflect the surface of the skin to create the physical 3D topological structure in real-time;
verifying the created physical 3D topological structure by measuring the deflections to the surface of the skin corresponding to physical 3D topological structure and comparing the measured deflections to the fed depth information; and
correcting any discrepancies between the measured deflections to the surface of the skin and the fed depth information by adjusting the pressurized air streams output by one or more air jets.

19. A system for creating a physical 3D topological structure from images, comprising:
a first depth camera that captures depth images of a structure to be replicated;
a processor that determines the depth information at a dense distribution of points corresponding to points in a depth image captured by the first camera;
actuators on sliding shafts in an array that receive the depth information captured by the first camera and adjust each sliding shaft to the depth in the depth image to create the physical 3D topological structure like the structure to be replicated; and
a second depth camera that captures depth images of the created physical 3D topological structure in order to verify that the created physical 3D topological structure accurately represents the structure to be replicated,
wherein discrepancies between the depth image of the surface to be replicated and the depth image of the created physical 3D topological surface are corrected by adjustments to the created physical 3D topological surface using the actuators.

20. The system of claim 19 wherein the created physical 3D topological surface changes when a new depth image is received by either the first depth camera or the second depth camera.

Assignees

• Microsoft Corp
• Microsoft Technology Licensing Llc

Inventors

• Therien Patrick
• Sinclair Michael

Classifications

• G06T15/00 Primary

  Three-dimensional [3D] image rendering · CPC title

• G09B5/06

  with both visual and audible presentation of the material to be studied · CPC title

• G06T15/005 Primary

  General purpose rendering architectures · CPC title

• G06T15/08

  Volume rendering · CPC title

• G09B21/003

  using tactile presentation of the information, e.g. Braille displays · CPC title