Information processing apparatus, information processing method, computer program, and image processing system

Patent metadata

Field	Value
Publication number	US-2017083084-A1
Application number	US-201515311341-A
Country	US
Kind code	A1
Filing date	Apr 21, 2015
Priority date	Jun 30, 2014
Publication date	Mar 23, 2017
Grant date	—

Abstract

Provided is an image display device configured to display a free viewpoint image that tracks a movement of observer's head and trunk.
The line-of-sight direction and trunk orientation of the user are individually oriented in a free viewpoint space displayed on a display device 500 based on posture information of the user's head and trunk, which are respectively obtained from a head motion tracking device 200 and a trunk motion tracking device 300. A behavior that is natural in real space can be represented by moving a line of sight to a front direction of the trunk on a space in a state where the line of sight does not match the front direction of the trunk. An unnatural UI that goes straight to the line-of-sight direction is prevented regardless of orientation of the trunk.

First claim

1 . An information processing apparatus comprising:
a receiver configured to receive first information on a posture of a head of an observer and second information on a posture of a body other than the head of the observer; and an image rendering processing unit configured to generate a display image corresponding to a posture of the observer based on the first information and the second information.

2 . The information processing apparatus according to claim 1 ,
wherein the receiver receives at least a posture of a trunk of the observer as the second information, and the image rendering processing unit generates a free viewpoint image that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on the first information and by orienting an orientation (viewpoint position) of the body of the observer in the free viewpoint space based on a posture of the trunk of the observer obtained from the second information.

3 . The information processing apparatus according to claim 2 ,
wherein the image rendering processing unit, when receiving a control signal used to instruct the observer to move in the free viewpoint space as an input, determines a point after movement (viewpoint position) by recognizing, as a front direction, an orientation of a body oriented based on a posture of the trunk of the observer obtained from the second information.

4 . The information processing apparatus according to claim 1 ,
wherein the image rendering processing unit generates a free viewpoint image that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on the first information and by arranging in a fixed position oriented based on the posture obtained from the second information.

5 . The information processing apparatus according to claim 1 ,
wherein the receiver receives at least a posture of the trunk of the observer as the second information, and the image rendering processing unit generates a free viewpoint image that tracks a posture of the head of the observer by orienting a line-of-sight direction of the observer in a free viewpoint space based on the first information, by orienting an orientation (viewpoint position) of the body of the observer in the free viewpoint space based on a posture of the trunk of the observer obtained from the second information, and by arranging a predetermined image widget in a fixed position oriented based on a posture of the trunk of the observer.

6 . The information processing apparatus according to claim 1 , further comprising:
a calibration processing unit configured to acquire a calibration parameter, wherein the image rendering processing unit performs image generation using posture information corrected by the calibration parameter.

7 . The information processing apparatus according to claim 6 ,
wherein the calibration processing unit, based on head posture information obtained from the first information and second posture information obtained from the second information, calculates a calibration parameter used to correct the second posture information and corrects the second posture information using the calibration parameter.

8 . The information processing apparatus according to claim 7 ,
wherein the calibration processing unit calculates a posture conversion parameter used to match the second posture information with the head posture information as the calibration parameter, by using the head posture information and the second posture information.

9 . The information processing apparatus according to claim 7 ,
wherein the posture information is represented by a quaternion, the calibration processing unit calculates a calibration quaternion used to correct the second posture information by multiplying the head posture information by an inverse of a quaternion of the second posture information from a right side, and the correction is performed by multiplying the quaternion of the second posture information by the calibration quaternion from a left side.

10 . The information processing apparatus according to claim 6 ,
wherein the calibration processing unit estimates a calibration parameter used to correct the second posture information based on rotational displacement for a certain period of a coordinate system of a head posture obtained from the first information and a coordinate system of a posture obtained from the second information and a property of a human body.

11 . The information processing apparatus according to claim 6 ,
wherein the calibration processing unit sets a time-series average of calibration parameters calculated for a certain period based on head posture information obtained from the first information and second posture information obtained from the second information to be a final calibration parameter.

12 . The information processing apparatus according to claim 11 ,
wherein the posture information is represented by a quaternion, and the calibration processing unit updates a calibration quaternion by performing spherical linear interpolation between a calibration quaternion newly calculated based on a quaternion of posture information obtained from each of the first information and the second information that have been newly received and a last calibration quaternion determined by the time-series average.

13 . The information processing apparatus according to claim 11 ,
wherein a posture of the trunk of the observer is calculated based on the second information, and the calibration processing unit determines an average value of an angular difference of a rotation direction, when a gravity direction is set to a rotation axis, of posture information of a head and a trunk obtained respectively from the first information and the second information that have been newly received, and calculates a calibration parameter on a gravity direction and the average value.

14 . The information processing apparatus according to claim 1 ,
wherein the image rendering processing unit processes a display image on the display unit based on a difference between a head posture obtained from the first information and a trunk posture obtained from the second information.

15 . The information processing apparatus according to claim 2 ,
wherein the image rendering processing unit generates a free viewpoint image by correcting a world coordinate system in a manner that a difference between a head posture and a trunk posture in the free viewpoint space decreases under a biosafety support mode.

16 . The information processing apparatus according to claim 2 ,
wherein the image rendering processing unit generates a free viewpoint image by fixing a head posture in a free viewpoint space at a point of time when a biosafety support mode is set and by changing a trunk posture in a free viewpoint space depending on a head posture detected by a head posture detection unit under the biosafety support mode.

17 . The information processing apparatus according to claim 15 ,
wherein the biosafety support mode is set when a state where a difference between a head posture obtained from the first information and a trunk posture obtained from the second information exceeds a first threshold continues for a certain time, and the biosafety support mode is released when the difference is equal to or less than a second threshold that is less than the first threshold.

18 . An information processing method comprising:
a receiving step of receiving first

Assignees

• Sony Corp

Inventors

• Tatsuta Hirokazu
• Kawamoto Kenta
• Koyama Yuichiro
• Kon Takayasu
• Hasegawa Yuichi
• Shimura Masaru
• Yamamoto Yuki
• Tanaka Eiichi

Classifications

• G06V40/103

  Static body considered as a whole, e.g. static pedestrian or occupant recognition · CPC title

• G06V40/161

  Detection; Localisation; Normalisation · CPC title

• G02B2027/0178

  Eyeglass type (eyeglass details G02C) · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title