Image generation apparatus and image generation method

Patent metadata

Field	Value
Publication number	US-9560274-B2
Application number	US-201114354671-A
Country	US
Kind code	B2
Filing date	Nov 7, 2011
Priority date	Nov 7, 2011
Publication date	Jan 31, 2017
Grant date	Jan 31, 2017

Abstract

Methods and apparatus provide for generating a component image in which, with images picked up using a front side camera and a rear side camera built in portable equipment, posture information of the portable equipment is associated; combining a plurality of component images between which the posture of the portable equipment is different to generate a synthesis image; changing over an image pickup mode between a double-sided synthesis image pickup mode in which a synthesis image of one half circumference picked up by the front side camera and a synthesis image of one half circumference picked up by the rear side camera are connected to each other to pick up a synthesis image of a whole circumference and a single-sided synthesis image pickup mode in which a synthesis image of a whole circumference is picked up by each of the front side camera and the rear side camera.

First claim

The invention claimed is:

1. An image generation apparatus, comprising:
a component image generation unit configured to generate a plurality of component images from a plurality of sequential images picked up using a plurality of cameras built in portable equipment, a first camera of the plurality of cameras being position toward a first side of a camera operator and a second camera of the plurality of cameras being positioned at least partially toward a second side of the camera operator, each of the plurality of cameras having optical axes different in at least one of a position and an orientation from each other, the component images including posture information of the portable equipment associated with the plurality of sequential images;
a synthesis image generation unit configured to combine the plurality of component images in which the posture of the portable equipment is different therebetween to generate a synthesis image; and
an image pickup mode changeover unit configured to change over an image pickup mode between: (i) a compound eye synthesis image pickup mode in which synthesis images of a first partial circumference picked up by the first camera of the plurality of cameras are connected to synthesis images of a second partial circumference picked up by the second camera of the plurality of cameras, and (ii) a single eye synthesis image pickup mode in which a single eye synthesis image is picked up using each of the plurality of cameras;
when the image pickup mode is the compound eye synthesis image pickup mode, the synthesis image generation unit: (i) combines a plurality of first component images picked up by the first of the plurality of cameras to generate a first side synthesis image, (ii) combines a plurality of second component images picked up by the second of the plurality of cameras to generate a second side synthesis image, and (iii) connects the first side synthesis image and the second side synthesis image to each other to generate a compound synthesis image.

2. The image generation apparatus according to claim 1 , wherein, when the image pickup mode is the single eye synthesis image pickup mode, the synthesis image generation unit combines a plurality of component images picked up by each of the cameras to generate a single eye synthesis image and combines and outputs the single eye synthesis images by the cameras into and as a stereo synthesis image or a plurality of synthesis images which provide different senses of distance.

3. The image generation apparatus according to claim 1 , further comprising:
an image correlation decision unit configured to decide a degree of correlation between the single eye synthesis images to be connected to each other in a boundary region in which the single eye synthesis images by the cameras are to be connected to each other;
when the degree of correlation is equal to or higher than a predetermined threshold value, the synthesis image generation unit connects the single eye synthesis images by the cameras to each other to generate a compound eye synthesis image.

4. The image generation apparatus according to claim 1 , further comprising:
a camera operator detection portion configured to detect catching of a camera operator in an image picked up by a front side camera;
a camera operator image pickup permission portion configured to issue an inquiry to a user about whether or not the catching of the camera operator is permitted; and
a region-of-interest setting portion configured to set, when the catching of the camera operator is permitted, based on an image of a facial expression of the camera operator picked up by a certain camera, a region-of-interest to an image picked up by a different camera;
the component image generation unit generating the component images such that the posture information of the portable equipment and the region-of-interest information set by the region-of-interest setting portion are associated with the image picked up using the different camera.

5. The image generation apparatus according to claim 1 , wherein the first camera is a front side camera positioned toward a front side of the camera operator and the second cameral is a rear side camera positioned toward a rear side of the camera operator;
the image pickup mode changeover unit changes over the image pickup mode between: (i) a double-sided synthesis image pickup mode in which synthesis images of one half circumference picked up by the front side camera and synthesis images of one half circumference picked up by the rear side camera are connected to each other to pick up a synthesis image of a whole circumference, and (ii) a single-sided synthesis image pickup mode in which a synthesis image of a whole circumference is picked up by each of the front side camera and the rear side camera; and
when the image pickup mode is the double-sided synthesis image pickup mode, the synthesis image generation unit combines a plurality of front side component images picked up by the front side camera to generate a front side synthesis image of one half circumference, combines a plurality of rear side component images picked up by the rear side camera to generate a rear side synthesis image of one half circumference, and then connects the front side synthesis image of the one half circumference and the rear side synthesis image of the one half circumference to each other to generate a synthesis image of a whole circumference.

6. The image generation apparatus according to claim 5 , wherein, when the image pickup mode is the single-sided synthesis image pickup mode, the synthesis image generation unit discards a plurality of front side component images picked up by the front side camera and combines a plurality of rear side component images picked up by the rear side camera to generate a rear side synthesis image of a whole circumference.

7. The image generation apparatus according to claim 5 , wherein, when the image pickup mode is the single-sided synthesis image pickup mode, the synthesis image generation unit combines a plurality of front side component images picked up by the front side camera to generate a front side synthesis image of a whole circumference, combines a plurality of rear side component images picked up by the rear side camera to generate a rear side synthesis image of a whole circumference, and combines and outputs the front side synthesis image of the whole circumference and the rear side synthesis image of the whole circumference into and as a stereo synthesis image or synthesis images of a near view and a distant view.

8. The image generation apparatus according to claim 5 , further comprising:
an image correlation decision unit configured to decide a degree of correlation between the front side synthesis image of the one half circumference and the rear side synthesis image of the one half circumference in a boundary region in which the front side synthesis image of the one half circumference and the rear side synthesis image of the one half circumference are connected to each other;
when the degree of correlation is equal to or higher than a predetermined threshold value, the synthesis image generation unit connects the front side synthesis image of the one half circumference and the rear side synthesis image of the one half circumference to each other to generate a synthesis image of the whole circumference.

9. The image generation apparatus according to claim 8 , further comprising an image pickup guide portion configured to guide the user, when the degree of correlation is lower than the predetermined threshold value, so as to carry out image pickup for further one half circumference.

10. The image generation apparatus according to any claim 5 , further comp

Assignees

• Hirata Shinichi
• Furumura Kyoko
• Yamagishi Takeshi
• Sony Corp
• Sony Interactive Entertainment Inc

Inventors

• Hirata Shinichi
• Furumura Kyoko
• Yamagishi Takeshi

Classifications

• H04N23/951

  by using two or more images to influence resolution, frame rate or aspect ratio · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G03B37/00 Primary

  Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe · CPC title

• H04N23/611

  where the recognised objects include parts of the human body · CPC title

• H04N23/667 Primary

  Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes · CPC title