Handling different input signals

Patent metadata

Field	Value
Publication number	US-10095032-B2
Application number	US-201414300649-A
Country	US
Kind code	B2
Filing date	Jun 10, 2014
Priority date	Jun 11, 2013
Publication date	Oct 9, 2018
Grant date	Oct 9, 2018

Abstract

A head mountable display (HMD) system in which the size and/or distance from the viewer of a virtual screen for display of video material depends upon one or more of the following:
whether the video material is in a 2-D or a 3-D format; the image resolution of the video material; the compression quality and/or data rate of the video material; the aspect ratio of the video material; the origin of the video material; and metadata associated with the video material.

First claim

We claim:

1. A head mountable display (HMD) system comprising:
an HMD having stereoscopic display elements for eyes of a wearer of the HMD, the stereoscopic display elements being configured to provide a virtual screen to the HMD wearer at a rendered virtual position within a three-dimensional display environment; and
a video processor configured to process video material for display by the stereoscopic display elements of the HMD;
in which the video processor is configured so that one or both of a size of the virtual screen and a distance from the wearer of the rendered virtual position of the virtual screen within the three-dimensional display environment for display of the video material by the HMD depends upon a spatial frequency content of the video material,
wherein when a maximum spatial frequency is less than a threshold amount, the system is configured to display the video material on a virtual screen that appears smaller and at a selected rendered virtual position which is closer to the wearer within the three-dimensional display environment than a virtual screen on which video with a maximum spatial frequency above the threshold amount is displayed.

2. A system according to claim 1 , comprising:
a detector configured to detect the spatial frequency content of the video material;
the video processor being configured to display the video material on a virtual screen having a size and a rendered virtual distance in the three-dimensional display environment from the wearer of the HMD according to the detected spatial frequency content of the video material.

3. A system according to claim 2 , in which, in a case where spatial content of the video material indicates a lower capture resolution than a current resolution of the video material, the video processor is configured to display the video material on a virtual screen having a size and a rendered virtual distance in the three-dimensional display environment from the wearer which depend on the capture resolution indicated by the detected spatial frequency content.

4. A system according to claim 1 , comprising an audio processor configured to generate audio content in a simulated audio presentation format dependent upon one or both of the size and a rendered virtual distance of the virtual screen in the three-dimensional display environment from the wearer selected by the video processor.

5. A system according to claim 4 , in which the HMD comprises a pair of earpieces; and in which the audio processor is configured to supply the generated audio content to the earpieces.

6. A system according to claim 4 , in which the audio processor is configured:
to simulate a single front central monophonic sound source in respect of audio content associated with video content displayed by a virtual screen of less than a first threshold virtual distance in the three-dimensional display environment from the wearer; and
to simulate multiple sound sources in respect of other audio content.

7. A system according to claim 1 , in which, in a case of display of 2-D material, one or both of a higher resolution and a higher data rate correspond to a virtual screen which has a rendered virtual position within the three-dimensional display environment which is more distant and which appears larger.

8. A system according to claim 1 , in which, in a case of 3-D material, the video material is displayed on a virtual screen of no more than 5 m diagonal and having a rendered virtual position within the three-dimensional display environment which is no more than 8 m from the user.

9. A system according to claim 1 , in which, in a case of 3-D material, the video material is displayed on a virtual screen of no more than 2 m diagonal and having a rendered virtual position within the three-dimensional display environment which is between 2 m and 3 m from the viewer.

10. A system according to claim 1 , in which, if metadata defining precautions against window violations is associated with the video material, a window violation occurring when an object in a 3-D environment approaches an edge of a display screen and, by virtue of its position in 3-D space and particular camera arrangements in use, the object appears in only one of a left or a right image but not the other, such metadata defining temporary changes to a lateral extent of the left and right images so as to mask objects which would otherwise give rise to window violations, then the metadata is used in rendering of 3-D video material.

11. A system according to claim 1 , in which, if metadata defining precautions against window violations is not associated with the video material, a window violation occurring when an object in a 3-D environment approaches an edge of a display screen and, by virtue of its position in 3-D space and particular camera arrangements in use, the object appears in only one of a left or a right image but not the other, such metadata defining temporary changes to a lateral extent of the left and right images so as to mask objects which would otherwise give rise to window violations, then the system is configured to analyse the video material and to change rendering of the video material according to the analysis.

12. A system according to claim 1 , in which the system is configured to detect head movements of the HMD in use, and in which the detection of translation as part of a head tracking detection is not used to vary a virtual viewpoint for video display.

13. A system according to claim 1 , the system being configured to transition gradually between different virtual screen sizes and different virtual screen distances from the viewer.

14. A video display system comprising:
a frame;
at least one a pair of stereoscopic display elements coupled to the frame to provide a three-dimensional display environment; and
a video processor arranged in operative communication with the pair of stereoscopic display elements,
in which the video processor is configured to generate video material for display, one or both of an image size and a rendered image position within the three-dimensional display environment at which the video material is displayed depending upon an image resolution of the video material and spatial frequency content of the video material, wherein if a maximum spatial frequency is less than a threshold amount, then the video material is configured by the video processor for display on a virtual screen associated with the pair of stereoscopic display elements, the virtual screen configured to appear smaller and at a selected rendered position within the three-dimensional display environment which is closer to a user than a virtual screen on which video with a maximum spatial frequency above the threshold amount is configured for display.

15. A method of operation of a stereoscopic head mountable display (HMD) system operable to provide a three dimensional display environment, comprising:
rendering a virtual screen for display of video material to a viewer of the HMD system, in which one or both of a size and rendered virtual distance from the viewer of the virtual screen within the three-dimensional display environment depends upon spatial frequency content of the video material;
determining, by one or more processors, whether a maximum spatial frequency is less than a threshold amount; and
when the maximum spatial frequency is determined to be less than the threshold amount, displaying the video material on a virtual screen that appears smaller and at a selected rendered virtual position within the three-dimensional display environment which is closer to the viewer than a virtual screen on which video

Assignees

• Sony Interactive Entertainment Europe Ltd

Inventors

• Bickerstaff Ian Henry
• Benson Simon Mark

Classifications

• G02B30/00

  Optical systems or apparatus for producing three-dimensional [3D] effects, e.g. stereoscopic images (in microscopes G02B21/22) · CPC title

• G02B27/017

  Head mounted · CPC title

• G02B2027/0134

  of stereoscopic type · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0127

  comprising devices increasing the depth of field · CPC title