Display

Patent metadata

Field	Value
Publication number	US-10054796-B2
Application number	US-201414778867-A
Country	US
Kind code	B2
Filing date	Mar 24, 2014
Priority date	Mar 25, 2013
Publication date	Aug 21, 2018
Grant date	Aug 21, 2018

Abstract

A display method is provided using a display that is operable to display an image to a viewer. The display method includes detecting one or both of an initial position and/or orientation of the viewer's head, and generating an image for display according to the detected position and/or orientation. A current position and/or orientation of the viewer's head are detected at a time at which the image is to be displayed. The method also includes re-projecting the generated image according to any differences between the initial position and/or orientation and the current position and/or orientation of the viewer's head, and displaying the re-projected image using the display.

First claim

The invention claimed is:

1. A display method using a display device operable to display an image to a viewer using a head-mountable display, the display method comprising:
detecting at least one of an initial position or orientation of the viewer's head;
rendering, by one or more processors, an image for display according to the at least one of the detected initial position or orientation to define a viewpoint of the viewer that is correct at a time that the rendering is initiated;
detecting at least one of a current position or orientation of the viewer's head depending upon a display time at which the rendered image is to be displayed, the at least one of the current position or orientation being used to define a viewpoint of the viewer appropriate to the display time;
re-projecting, by one or more processors, the rendered image according to any differences between the at least one of the detected initial position and orientation and the at least one of the current position or orientation of the viewer's head, to perform a viewpoint adjustment of the rendered image to the viewpoint of the viewer appropriate to the display time; and
displaying the re-projected image using the head-mountable display at the display time;
in which the step of re-projecting comprises detecting an overlapping portion between the rendered image and the re-projected image, and reproducing the overlapping portion as part of the re-projected image, wherein the method further comprises filling portions of the re-projected image other than the overlapping portion with image material from a further image source.

2. The method according to claim 1 , in which the rendering step comprises associating metadata with the rendered image, the metadata indicating the at least one of the initial position or orientation of the viewer's head.

3. The method according to claim 1 , in which the rendering step comprises providing depth data indicating image depth of one or more image features, and the step of re-projecting comprises repositioning selected ones of the one or more image features within the re-projected image according to the depth data.

4. The method according to claim 1 , in which the at least one of the position or orientation of the viewer's head is detected by detecting at least one of a position or orientation of the head-mountable display.

5. The method according to claim 1 , in which the rendering step takes a period of time equal to at least an image generating latency, and the re-projecting step takes a period of time equal to at least an image re-projection latency, the image generating latency being longer than the image re-projection latency.

6. The method according to claim 5 , in which the detected initial position is detected at a latest time, allowing for the image generating latency and the image re-projection latency, to allow for the display of the re-projected image at the display time.

7. A display method for displaying an image to a viewer, the method comprising:
receiving an image and associated metadata, wherein:
the image is captured using a camera,
at least one of the position or orientation of the camera is detected, and
the metadata is associated with the captured image, the metadata indicating the at least one of the position or orientation of the camera at a time of capture of the image;

detecting at least one of a current position or orientation of the viewer's head at a time at which the image is to be displayed;
re-projecting, by at least one processor, the received image according to any differences between the at least one of the position or orientation indicated by the metadata and the at least one of the current position or orientation of the viewer's head; and
displaying the re-projected image;
in which the re-projecting comprises detecting an overlapping portion between the received image and the re-projected image, and reproducing the overlapping portion as part of the re-projected image, wherein the method further comprises filling portions of the re-projected image other than the overlapping portion with image material from a further image source.

8. The method according to claim 7 , comprising the step of combining the re-projected image with a panoramic image representing a panoramic field of view from the position of the camera.

9. A display method for displaying an image to a viewer, the method comprising:
receiving an insertion television image and associated metadata indicating at least one of a position or orientation of a television camera at a time of capture or generation of the insertion television image;
re-projecting, by at least one processor, the received television insertion image to a display position dependent upon a current orientation of the camera, the display position being relative to a panoramic image representing a panoramic field of view from that camera position;
generating an output image for display to the viewer by replacing a part of the panoramic image with the re-projected insertion television image according to the display position dependent upon the current orientation of the camera; and
displaying the output image to the viewer.

10. The method according to claim 8 , further comprising selecting a panoramic image according to the camera used or the camera position of the image.

11. The method according to claim 8 , further comprising removing graphical data from the image.

12. The method according to claim 11 , further comprising inserting graphical information for display after re-projection of the image.

13. The method according to claim 7 , in which the image is displayed using a head-mountable display and the at least one of the position or orientation of the viewer's head is detected by detecting at least one of a position or orientation of the head-mountable display.

14. A non-transitory computer-readable storage medium storing computer readable instructions there, the instructions, when executed by one or more processors, cause the processors to perform a display method, the method comprising:
detecting at least one of an initial position or orientation of a viewer's head;
rendering an image for display according to the at least one of the detected initial position or orientation to define a viewpoint of the viewer that is correct at a time that the rendering is initiated;
detecting at least one of a current position or orientation of the viewer's head depending upon a display time at which the rendered image is to be displayed, the at least one of the current position or orientation being used to define a viewpoint of the viewer appropriate to the display time;
re-projecting the rendered image according to any differences between the at least one of the initial position and orientation and the at least one of the current position or orientation of the viewer's head to perform a viewpoint adjustment of the rendered image to the viewpoint of the viewer appropriate to the display time; and
displaying the re-projected image to the viewer at the display time using a head-mountable display;
in which the re-projecting comprises detecting an overlapping portion between the rendered image and the re-projected image, and reproducing the overlapping portion as part of the re-projected image, wherein the method further comprises filling portions of the re-projected image other than the overlapping portion with image material from a further image source.

15. A head-mountable display system comprising:
a display element observable by a viewer;
a detector device configured to detect at least one of a position or orientation of the viewer's head; and
an

Assignees

• Sony Interactive Entertainment Europe Ltd

Inventors

• Bickerstaff Ian Henry
• Benson Simon Mark
• Wright Oliver Mark
• Raghoebardajal Sharwin Winesh

Classifications

• H04N23/683

  performed by a processor, e.g. controlling the readout of an image memory · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/6811

  based on the image signal · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title