Device, method, and computer program for three-dimensional video processing

Patent metadata

Field	Value
Publication number	US-9602799-B2
Application number	US-201213365858-A
Country	US
Kind code	B2
Filing date	Feb 3, 2012
Priority date	Jan 14, 2011
Publication date	Mar 21, 2017
Grant date	Mar 21, 2017

Abstract

A 3D video processing device includes a disparity map generator which compares a first-perspective image from a first perspective with a second-perspective image from a second perspective, which are two images forming stereoscopic video, thereby generates a disparity map which represents disparity of the second-perspective image with respect to the first-perspective image, a first new image generator which rotates the first-perspective image based on rotation angle information which represents rotation statuses of the first- and the second-perspective images, thereby generates a new first-perspective image, and a second new image generator which generates a new second-perspective image, which is paired with the new first-perspective image to form stereoscopic video, based on the first-perspective image and on the disparity map.

First claim

What is claimed is:

1. A video processing device comprising:
an input device for obtaining a first input image of a first perspective and a second input image of a second perspective different from the first perspective, the first input image and the second input image having a rectangular shape; and
an output device for outputting a first output image of the first perspective and a second output image of the second perspective, the first output image and the second output image having the rectangular shape,
wherein the video processing device is configured to:
generate a first disparity map, which has the rectangular shape, from the first input image and the second input image; and
when the first output image is generated by rotating an orientation of the first input image by an angle in the rectangular shape, generate a second disparity map, which has the rectangular shape, by rotating an orientation of the first disparity map by the angle within the rectangular shape, and generate pixels of the second output image by shifting pixels of the first output image in a longer side direction in which a longer side of the rectangular shape extends, based on the second disparity map.

2. The video processing device according to claim 1 , further comprising a vertical sensor configured to detect the angle which is between a short side direction of the rectangular shape and an orientation of a vertical direction in the video processing device.

3. The video processing device according to claim 1 , wherein:
the input device comprises a first image sensor and a second image sensor,
the first image of the first perspective is captured by the first image sensor, and
the second image of the second perspective is captured by the second image sensor.

4. The video processing device according to claim 1 , wherein the output device comprises a display.

5. The video processing device according to claim 1 , wherein the output device comprises a memory card slot.

6. The video processing device according to claim 1 , wherein the first image of the first perspective is for a right eye of a user, and the second image of the first perspective is for a left eye of the user.

7. The video processing device according to claim 1 , wherein the first image of the first perspective is for a left eye of a user, and the second image of the first perspective is for a right eye of the user.

8. The video processing device according to claim 1 , wherein:
each of the first input image, the second input image, and the first disparity map comprises sub-regions, and
a disparity amount of each sub-region of the first disparity map is calculated with a block matching process between the sub-regions of the first input image and the corresponding sub-regions of the second input image.

9. The video processing device according to claim 8 , wherein each sub-region of the sub-regions of the first input image, the second input image, and the first disparity map comprises 16×16 pixels.

10. The video processing device according to claim 8 , wherein the pixels of the second output image is generated by shifting the pixels of the first output image in the longer side direction of the rectangular shape, based on a disparity amount of each sub-region of the second disparity map.

11. A video processing method for a video processing device,
the video processing device comprising:
an input device for obtaining a first input image of a first perspective and a second input image of a second perspective being different from the first perspective, the first input image and the second input image having a rectangular shape; and
an output device for outputting a first output image of the first perspective and a second output image of the second perspective,

the video processing method comprising:
generating a first disparity map, which has the rectangular shape, from the first input image and the second input image; and
when the first output image, which has the rectangular shape, is rotated at an angle from the first input image, generating a second disparity map by rotating, by the angle, the first disparity map within the rectangular shape, and generating pixels of the second output image by shifting pixels of the first output image in a longer side direction in which a longer side of the rectangular shape extends, based on the second disparity map.

12. The video processing method according to claim 11 , wherein:
the video processing device further comprises a vertical sensor configured to detect the angle which is between a short side direction of the rectangular shape and an orientation of a vertical direction in the video processing device, and
the method further comprises detecting the angle by the vertical sensor.

13. The video processing method according to claim 11 , wherein:
the input device comprises a first image sensor and a second image sensor, and
the method further comprises:
capturing the first image of the first perspective by the first image sensor; and
capturing the second image of the second perspective by the second image sensor.

14. The video processing method according to claim 11 , wherein the output device comprises a display.

15. The video processing method according to claim 11 , wherein the output device comprises a memory card slot.

16. The video processing method according to claim 11 , wherein the first image of the first perspective is for a right eye of a user, and the second image of the first perspective is for a left eye of the user.

17. The video processing method according to claim 11 , wherein the first image of the first perspective is for a left eye of a user, and the second image of the first perspective is for a right eye of the user.

18. The video processing method according to claim 11 , wherein:
each of the first input image, the second input image, and the first disparity map comprises sub-regions, and
a disparity amount of each sub-region of the first disparity map is calculated with a block matching process between the sub-regions of the first input image and the sub-regions of the second input image.

19. The video processing method according to claim 18 , wherein each sub-region of the sub-regions of the first input image, the second input image, and the first disparity map comprises 16×16 pixels.

20. The video processing method according to claim 18 , wherein the pixels of the second output image is generated by shifting the pixels of the first output image in the longer side direction of the rectangular shape, based on a disparity amount of each sub-region of the second disparity map.

Assignees

• Kobayashi Yuki
• Maruyama Yuki
• Matsumoto Kentaro
• Panasonic Ip Man Co Ltd

Inventors

• Kobayashi Yuki
• Maruyama Yuki
• Matsumoto Kentaro

Classifications

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• H04N2013/0081

  Depth or disparity estimation from stereoscopic image signals · CPC title

• H04N13/341

  using temporal multiplexing · CPC title

• H04N13/239

  using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular distance (H04N13/243 takes precedence) · CPC title

• H04N13/0438

  Electricity · mapped topic