Projection and camera system for augmented reality environment

Patent metadata

Field	Value
Publication number	US-9472005-B1
Application number	US-201213527503-A
Country	US
Kind code	B1
Filing date	Jun 19, 2012
Priority date	Apr 18, 2012
Publication date	Oct 18, 2016
Grant date	Oct 18, 2016

Abstract

A device includes a projection and camera system to create an augmented reality environment in which images are projected onto a scene and user movement within the scene is captured. The projection and camera system have a camera to image scattered IR light from the scene and compute time of flight values used in depth mapping of objects in the room. The system also has a projector to project the images onto the scene. The projected images and scattered IR light use a common optical path through the same lens. Additionally, an illumination system used to illuminate the scene with the IR light may also use the common optical path through the same lens. In one implementation, the projection and camera system are mounted in a movable head of a table lamp.

First claim

What is claimed is:

1. A device comprising:
a head structure;
a base structure connected to the head structure by an arm mechanism;
a plurality of components housed in one or more of the head structure or the base structure, the plurality of components comprising:
a lens;
a projector to project images, through the lens, onto one or more objects in an environment external to the device;
an illumination component to illuminate, through the lens, the one or more objects in the environment into which at least a portion of the images from the projector are to be projected;
a time-of-flight sensor to receive at least a portion of light from the illumination component scattered by the environment, the light being received at the lens, the time-of-flight sensor configured to determine a time of flight value associated with one or more objects associated with the one or more objects in the environment; and
a processor to determine distance information of the one or more objects relative to the time of flight sensor, the distance information determined based at least in part on the determined time of flight value.

2. The device of claim 1 , wherein the illumination component, the lens, the time-of-flight sensor, and the projector are arranged within the head structure.

3. The device of claim 1 , wherein the illumination component, the lens, the time-of-flight sensor, and the projector are arranged within the head structure and the processor resides in the base structure.

4. The device of claim 1 , wherein the illumination component comprises an infrared (IR) device to generate IR light.

5. The device of claim 1 , wherein the illumination component generates an illumination field and the projector generates a projector field, the illumination field being separate from the projector field.

6. The device of claim 1 , wherein the projector projects an image that covers a first area of the scene, and the illumination component illuminates a second area of the scene, wherein the second area is larger than the first area.

7. The device of claim 1 , wherein the head structure is movably connected to the arm mechanism via a connector that permits rotation about two different axes.

8. The device of claim 1 , wherein the head structure is movably connected to the arm mechanism via a connector that permits rotation about a pan axis and a tilt axis.

9. The device of claim 1 , further comprising:
one or more speakers arranged in the base structure; and
a power component to provide power to at least the processor, the power component residing in the base structure.

10. The device of claim 1 , further comprising one or more microphones, the one or more microphones being arranged in at least one of the base structure or the head structure.

11. A system comprising:
a lens;
a projector to project an image through the lens into an area external to the system;
an illumination component, independent of the projector, to illuminate through the lens an area external to the system and into which at least a portion of the image from the projector is being projected; and
a time-of-flight component to receive light scattered from the illuminated area and determine a time-of-flight value associated with one or more objects in the illuminated area, the light being received back through the lens.

12. The system of claim 11 , wherein the lens comprises a zoom lens.

13. The system of claim 11 , wherein the illumination component comprises one of an infrared (IR) device to generate IR light and a plurality of infrared (IR) light emitting diodes (LEDs) positioned adjacent the lens.

14. The system of claim 11 , wherein the illumination component generates an illumination field and the projector generates a projector field, the illumination field being separate from the projector field.

15. The system of claim 11 , further comprising a beam splitter to reflect the projected image from a first path onto the optical path, the beam splitter further passing the collected light along the optical path without reflection.

16. The system of claim 11 , further comprising a beam splitter to pass the projected image along the optical path and to reflect the collected light from the optical path onto a first path.

17. The system of claim 11 , wherein the light from the illumination component comprises infrared (IR) light, and further comprising:
an IR filter; and
a second lens to direct the received IR light onto the time-of-flight component through the IR filter.

18. A system comprising:
a lens;
a projector to project an image through the lens;
an illumination component to generate light that is directed out through the lens to illuminate an environment external to the system and into which at least a portion of the image from the projector is being projected; and
a time-of-flight component to receive light scattered from the illuminated area and determine a time-of-flight value associated with one or more objects in the illuminated area, the light being received through the lens.

19. The system of claim 18 , wherein the lens comprises a zoom lens.

20. The system of claim 18 , wherein the illumination component comprises an infrared (IR) device to generate IR light.

21. The system of claim 18 , further comprising a beam splitter to reflect the projected image from a first path onto the lens, the beam splitter further passing the collected light onto the lens without reflection.

22. The system of claim 18 , further comprising a beam splitter to pass the projected image onto the lens and to reflect the collected light from the lens onto a first path.

23. The system of claim 18 , wherein the light from the illumination component comprises infrared (IR) light, and further comprising:
an IR filter; and
a second lens to direct the received IR light onto the time-of-flight component through the IR filter.

24. A system comprising:
a lens having an associated optical path;
a projector to project an image through the lens along at least a portion of the optical path into an environment external to the system;
an illumination component to project light through the lens along at least a portion of the optical path in order to illuminate the environment into which at least a portion of the image from the projector is being projected; and
a sensor to receive light scattered from the environment through the lens along at least a portion of the optical path, the sensor to use the received light for analysis of one or more objects in the environment.

25. The system of claim 24 , wherein the lens comprises a zoom lens.

26. The system of claim 24 , wherein the sensor comprises a time-of-flight sensor to image the received light for use in computing a time of flight value associated with the one or more objects in the environment.

27. The system of claim 24 , wherein the illumination component is a laser.

28. The system of claim 24 , wherein the light generated by the illumination component is the light that is scattered by the environment.

29. A method comprising:
illuminating an environment with infrared (IR) light;
capturing, through a lens, the IR light scattered by one or more objects in the environment;
determining a time of flight value based on the scattered IR light that is captured through the lens; and
projecting an image through the lens into the environment be

Assignees

• Marason Eric G
• Weatherford William Thomas
• Coley Christopher D
• Yuan Robert A
• Virgen Miguel
• Amazon Tech Inc

Inventors

• Marason Eric G
• Weatherford William Thomas
• Coley Christopher D
• Yuan Robert A
• Virgen Miguel

Classifications

• H04N9/3179

  Video signal processing therefor · CPC title

• G01B11/25

  by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50}) · CPC title

• G01S17/86

  Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders · CPC title

• G01S17/894

  Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar · CPC title

• H04N9/3194

  including sensor feedback · CPC title