Methods, systems, and computer readable media for generating an augmented scene display

Patent metadata

Field	Value
Publication number	US-9858721-B2
Application number	US-201414761229-A
Country	US
Kind code	B2
Filing date	Jan 15, 2014
Priority date	Jan 15, 2013
Publication date	Jan 2, 2018
Grant date	Jan 2, 2018

Abstract

The subject matter described herein includes systems, methods, and computer readable media for generating an augmented scene display. An exemplary method includes forming, using a display device operating in a first stage, an augmented virtual image by emitting light rays through a plurality of spatial light modulation layers included in a display device. The method also includes forming, using the display device operating in a second stage, an occluded real image by opening a shutter element of the display device to receive light rays from a real object and utilizing the plurality of spatial light modulation layers to block any light ray from the real object which coincides with the augmented virtual image. The method further includes generating an augmented scene display that includes both the occluded real image and the augmented virtual image by alternating the operation of the display device between the first stage and the second stage.

First claim

What is claimed is:

1. A method for generating an augmented scene display, the method comprising:
forming, using a display device operating in a first stage, an augmented virtual image by emitting light rays through a plurality of spatial light modulation layers included in the display device;
forming, using the display device operating in a second stage, an occluded real image by opening a shutter element of the display device to receive light rays from a real object and utilizing the plurality of spatial light modulation layers to block any light ray from the real object which coincides with the augmented virtual image; and
generating an augmented scene display that includes both the occluded real image and the augmented virtual image by alternating the operation of the display device between the first stage and the second stage.

2. The method of claim 1 wherein the plurality of spatial light modulation layers includes a plurality of transparent liquid crystal displays (LCDs).

3. The method of claim 1 wherein the display device includes a head-worn device.

4. The method of claim 1 wherein the plurality of spatial light modulation layers is configured to attenuate the emitted light rays.

5. The method of claim 1 wherein the emitted light rays used to form an augmented virtual image are generated by a backlight element included in the display device.

6. The method of claim 5 wherein the first stage includes a virtual image formation phase wherein the light rays are emitted from a backlight element in the display device and passes through patterns displayed on the plurality of spatial light modulation layers to form the augmented virtual image while the light rays from the real object are blocked by the shutter element.

7. The method of claim 6 wherein the second stage includes an occluded real-image formation phase wherein the shutter element is opened and a backlight element in the display device is disabled.

8. The method of claim 1 wherein the shutter element is configured to be switched from a globally opaque state in the first stage to a globally transparent state in the second stage.

9. The method of claim 8 wherein the shutter element includes a single liquid crystal cell.

10. The method of claim 5 wherein the backlight includes a transparent organic light emitting diode (OLED) panel.

11. The method of claim 1 wherein the first stage and second state are alternated beyond a flicker fusion threshold.

12. The method of claim 1 wherein the plurality of spatial light modulation layers are constrained to allow a focused image to be formed on an eye or a camera when the display device is positioned closer than the focal distance of the eye or the camera.

13. A system for generating an augmented scene display, the system comprising:
a display device that includes:
a backlight element configured to emit light rays in a first stage;
a shutter element configured to open and allow light rays from a real object to be received by the display device in a second stage; and
a plurality of spatial light modulation layers configured to form, in the first stage, an augmented virtual image by receiving and attenuating the emitted light rays and to form, in the second stage, an occluded real object image by blocking any light ray from the real object that coincides with the augmented virtual image, and to generate an augmented scene display that includes both the occluded real image and the augmented virtual image by alternating between the first stage and the second stage.

14. The system of claim 13 wherein the plurality of spatial light modulation layers includes a plurality of transparent liquid crystal displays (LCDs).

15. The system of claim 13 wherein the display device includes a head-worn device.

16. The system of claim 13 wherein the emitted light rays used to form an augmented virtual image are generated by the backlight element.

17. The system of claim 16 wherein the first stage includes a virtual image formation phase wherein the light rays emitted from the backlight element pass through patterns displayed on the plurality of spatial light modulation layers to form the augmented virtual image while the light rays from the real object are blocked by the shutter element.

18. The system of claim 17 wherein the second stage includes an occluded real-image formation phase wherein the shutter element is opened and the backlight element is disabled.

19. The system of claim 13 wherein the shutter element is configured to be switched from a globally opaque state in the first stage to a globally transparent state in the second stage.

20. The system of claim 19 wherein the shutter element includes a single liquid crystal cell.

21. The system of claim 13 wherein the backlight includes a transparent organic light emitting diode (OLED) panel.

22. The system of claim 13 wherein the first stage and second state are alternated beyond a flicker fusion threshold.

23. The system of claim 13 wherein the plurality of spatial light modulation layers are constrained to allow a focused image to be formed on an eye or a camera when the display device is positioned closer than the focal distance of the eye or the camera.

24. A method for generating an augmented scene display, the method comprising:
forming, using a display device operating in a first stage, an augmented virtual image by emitting light rays from a transparent light emissive display through a first plurality of spatial light modulation layers included in front of the transparent light emissive display, wherein the light rays are emitted by the transparent light emissive display only in display regions corresponding to the augmented virtual image;
forming, using the display device operating in a second stage, an occluded real image by setting to an initial clear state a second plurality of spatial light modulation layers of the display device located behind the transparent light emissive display to receive light rays from a real object and utilizing at least one of the second plurality of spatial light modulation layers to block any light ray from the real object which coincides with the augmented virtual image; and
generating an augmented scene display that includes both the occluded real image and the augmented virtual image by conducting the first stage and the second stage contemporaneously.

25. The method of claim 24 wherein the transparent light emissive display includes a transparent organic light emitting diode (OLED) panel.

26. A system for generating an augmented scene display, the system comprising:
a display device that includes:
a transparent light emissive display configured to emit light rays in a first stage;
a first plurality of spatial light modulation layers configured to form, in the first stage, an augmented virtual image by receiving and attenuating the emitted light rays;
a second plurality of spatial light modulation layers configured to, in a second stage, an initial clear state to allow light rays from a real object to be received by the display device in a second stage and to form an occluded real object image by blocking any light ray from the real object that coincides with the augmented virtual image; and
wherein the display device generates an augmented scene display that includes both the occluded real image and the augmented virtual image by conducting the first stage and the second stage contemporaneo

Assignees

• Univ North Carolina Chapel Hill

Inventors

• Maimone Andrew
• Fuchs Henry

Classifications

• G02B27/017

  Head mounted · CPC title

• G02B27/0172 Primary

  characterised by optical features · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title