Video outputting apparatus, three-dimentional video observation device, video presentation system, and video outputting method

Patent metadata

Field	Value
Publication number	US-9741168-B2
Application number	US-201314405928-A
Country	US
Kind code	B2
Filing date	Apr 18, 2013
Priority date	Jun 29, 2012
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

An object classification unit 504 classifies objects placed in a virtual three-dimensional space into objects to be displayed on an optical transmission type HMD and objects to be displayed on a monitor 300 which is observed through the optical transmission type HMD. A video outputting unit 506 outputs a video of each object to the monitor 300 or the optical transmission type HMD based on a classification by the object classification unit 504.

First claim

The invention claimed is:

1. A video outputting apparatus, comprising:
an object classification unit configured to classify objects placed in a virtual three-dimensional space into objects to be displayed on an optical transmission type HMD (head mounted display) and objects to be displayed on a three-dimensional (3D) monitor which is observed through an optical shutter in the optical transmission type HMD,
wherein the object classification unit classifies the objects based upon a distance of each object measured from the 3D monitor,
wherein, if the distance of an object is less than a predetermined value, the object is displayed on the 3D monitor, and
wherein, if the distance of the object is greater than or equal to the predetermined value, the object is displayed on the optical transmission type HMD; and a video outputting unit configured to output a video of the objects to the 3D monitor or the optical transmission type HMD based on the classification by the object classification unit,
wherein the virtual three-dimensional space has an orthogonal coordinate system of an x axis, a y axis, and a z axis set thereto, and an xy plane defined by the x axis and the y axis extends in parallel to a display area of the 3D monitor,
wherein the classification of each object is reevaluated for each frame of the video such that display of each object transitions from the optical transmission type HMD if a z-distance of the object becomes less than the predetermined value and transitions from the 3D monitor to the optical transmission type HMD if the z-distance of the object becomes equal to or greater than the predetermined value.

2. The video outputting apparatus according to claim 1 , wherein the object classification unit classifies an object determining the coordinate of a center of gravity of a plurality of polygons which configure the object as the coordinate of the object.

3. The video outputting apparatus according to claim 1 , wherein the optical transmission type HMD includes an image pickup element for picking up an image of a plurality of image pickup objects existing in a region including a field of view of a user who wears the optical transmission type HMD,
the objects placed in the virtual three-dimensional space include a plurality of AR (Augmented Reality) images, each AR image being associated with a unique pickup object from the plurality of pickup objects,
wherein each AR image is produced and located in the virtual three-dimensional space using the associated unique pickup object as a marker, and
the object classification unit classifies the plurality of AR images as objects only to be displayed on the optical transmission type HMD.

4. The video outputting apparatus according to claim 1 , wherein the objects placed in the virtual three-dimensional space include an operation target object whose position coordinate in the virtual three-dimensional space can be changed in response to an operation of the user who wears the optical transmission type HMD, and
the object classification unit classifies the operation target object as an object to be displayed on the optical transmission type HMD.

5. A video presentation system, comprising:
a three-dimensional (3D) monitor of a frame sequential type;
a three-dimensional video observation device including an optical transmission type HMD (head mounted display) and an optical shutter to be used for observation of the three-dimensional 3D monitor;
an object classification unit configured to classify objects placed in a virtual three-dimensional space into objects to be displayed on the optical transmission type HMD and objects to be displayed on the three-dimensional 3D monitor;
wherein the virtual three-dimensional space has an orthogonal coordinate system of an x axis, a y axis, and a z axis set thereto, and an xy plane defined by the x axis and the y axis extends in parallel to a display area of the 3D monitor,
wherein the object classification unit classifies the objects based upon a z-distance of each object measured from the 3D monitor,
wherein, if the z-distance of an object is less than a predetermined value, the object is displayed on the 3D monitor, and
wherein, if the z-distance of the object is greater than or equal to the predetermined value, the object is displayed on the optical transmission type HMD; and
a video outputting unit configured to output a video of the objects to the 3D monitor or the optical transmission type HMD based on the classification by the object classification unit,
wherein the classification of each object is reevaluated for each frame of the video such that display of each object transitions from the optical transmission type HMD if the z-distance of the object becomes less than the predetermined value and transitions from the 3D monitor to the optical transmission type HMD if the z-distance of the object becomes equal to or greater than the predetermined value.

6. The video presentation system according to claim 5 , wherein the object classification unit further classifies an object based on a relationship in magnitude between a display classification reference speed value determined in order to classify an object regarding whether the object is to be displayed on the 3D monitor or the optical transmission type HMD and a speed of change of a z-coordinate of the object.

7. The video presentation system according to claim 6 , wherein the object classification unit classifies an object in the virtual three-dimensional space as an object to only be displayed on the optical transmission type HMD and not on the 3D monitor when the speed of change of the z-coordinate of the object is higher than the display classification reference speed value.

8. A video outputting method, comprising:
classifying objects placed in a virtual three-dimensional space based upon a z-distance of each object measured from a 3D monitor into objects to be displayed on an optical transmission type HMD (head mounted display) and objects to be displayed on the 3D monitor which is observed through an optical shutter in the optical transmission type HMD by a user who wears the optical transmission type HMD,
wherein the virtual three-dimensional space has an orthogonal coordinate system of an x axis, a y axis, and a z axis set thereto, and an xy plane defined by the x axis and the y axis extends in parallel to a display area of the 3D monitor,
wherein, if the z-distance of an object is less than a predetermined value, the object is displayed on the 3D monitor, and
wherein, if the z-distance of the object is greater than or equal to the predetermined value, the object is displayed on the optical transmission type HMD; and
outputting a video of each object to the 3D monitor or the optical transmission type HMD based upon the classification of each object,
wherein the classification of each object is reevaluated for each frame of the video such that display of each object transitions from the optical transmission type HMD if the z-distance of the object becomes less than the predetermined value and transitions from the 3D monitor to the optical transmission type HMD if the z-distance of the object becomes equal to or greater than the predetermined value.

9. A non-transitory computer readable medium having stored thereon a program for a computer, the program comprising:
classifying objects placed in a virtual three-dimensional space based upon a z-distance of each object measured from a 3D monitor into objects to be displayed on an optical transmission type HMD (head mounted display) and objects to be displayed on the 3D monitor which is observed through an optical shutter in the optical transmission type HMD by a user who wears the optical transmission type HMD,
where

Assignees

• Sony Computer Entertainment Inc
• Sony Corp
• Sony Interactive Entertainment Inc

Inventors

• Ohashi Yoshinori

Classifications

• G06F3/1423 Primary

  controlling a plurality of local displays, e.g. CRT and flat panel display · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G09G2320/0261

  in the context of movement of objects on the screen or movement of the observer relative to the screen · CPC title

• G09G2300/023

  Display panel composed of stacked panels · CPC title

• G09G3/003

  to produce spatial visual effects · CPC title