Image processing device, image processing method, non-transitory computer readable recording medium and photographing assist equipment

Patent metadata

Field	Value
Publication number	US-10713828-B2
Application number	US-201715422476-A
Country	US
Kind code	B2
Filing date	Feb 2, 2017
Priority date	Feb 2, 2016
Publication date	Jul 14, 2020
Grant date	Jul 14, 2020

Abstract

An image processing device for stitching a plurality of input images together so as to generate a panoramic composite image is provided. An imaging control section sets a guide for photographing a photographing subject from different imaging directions as a guide information for obtaining a plurality of input images suitable for the image composition. A guide display setting section displays a photographing guide image based on the guide information. A user rotates a camera while checking the guide image and captures the plurality of input images from different imaging directions. An image compositing section stitches the plurality of input images together so as to generate a composite image.

First claim

What is claimed is:

1. An image processing device, comprising:
a display device configured to display a photographing subject;
an imaging device configured to photograph the photographing subject;
a first motion sensor configured to detect a rotational direction and a degree of rotation of the imaging device; and
a processor electrically connected to the display device, the image device and the first motion sense and configured to:
display a guide on the display device, for photographing the photographing subject from different imaging directions as the guide indicates the rotational direction of the imaging device relative to a proper rotational direction for photographing a next image of the photographing subject, wherein display the guide on the display device further comprising changing a display manner of the guide according to the rotational direction and the degree of rotation of the imaging device,

display a supplementary guide simultaneously with the guide for reducing a change in a photographing position on the display device by notifying a deviation of the imaging device in a vertical direction and in a horizontal direction,
photograph the next image as the guide and the supplementary guide are vertically aligned in the vertical direction of the display device of the image processing device, and stitch together a plurality of images which comprises the next image obtained by photographing the photographing subject from the different imaging directions so as to generate a composite image, wherein the supplementary guide comprises a third image displayed such that a display location does not change according to the direction and the amount of a parallel shift of the imaging device and a fourth image displayed such that a display location changes according to the direction and the amount of the parallel shift of the imaging device.

2. The image processing device according to claim 1 , wherein the processor further configures the guide to indicate rotating the imaging device to photograph the photographing subject from the different imaging directions.

3. The image processing device according to claim 1 , wherein the guide comprises the first image indicating a total imaging area that is an area to be photographed into the plurality of images for generating the composite image.

4. The image processing device according to claim 3 , wherein the first image further indicates the number of the plurality of images required to photograph the total imaging area.

5. The image processing device according to claim 3 ,
wherein the guide further comprises a second image indicating a current imaging area, and
the processor is further configured to change a location of the second image relative to the first image according to the rotational direction and the degree of rotation of the imaging device.

6. The image processing device according to claim 5 , wherein the location of the second image relative to the first image indicates the imaging direction with reference to the imaging device.

7. The image processing device according to claim 3 ,
wherein the guide further comprises a reduced image of a photographed image, and
the processor is further configured to change a location of the reduced image relative to the first image according to the rotational direction and the degree of rotation of the imaging device.

8. The image processing device according to claim 5 , wherein the location of the second image relative to the first image indicates the imaging direction with reference to a user.

9. The image processing device according to claim 5 , wherein when the location of the second image relative to the first image indicates the imaging direction, an image displayed on the display device when indicating the imaging angle with reference to the imaging device and an image displayed on the display device when indicating the imaging angle with reference to a user are horizontally flipped images of each other.

10. The image processing device according to claim 5 , wherein the first image is sectioned into a plurality of blocks that correspond to the respective plurality of images required to photograph the total imaging area, and the location of the second image over any one of the blocks indicates the imaging direction.

11. The image processing device according to claim 10 , wherein for each of the plurality of blocks, the first image indicates a photographing order corresponding to the block.

12. The image processing device according to claim 3 , wherein
the processor is further configured to:
detect a face area of a person displayed on the display device, and
set the first image so that the total imaging area is centered on the face area detected by the processor.

13. The image processing device according to claim 3 , wherein
the processor is further configured to detect a face area of a person displayed on the display device, and
when the processor detects a plurality of face areas, the processor is further configured to set the first image so that the total imaging area is centered on one of the detected plurality of face areas.

14. The image processing device according to claim 3 , wherein
the processor is further configured to detect a face area of a person displayed on the display device, and
when the processor detects a plurality of face areas, the processor is further configured to set the first image so that the total imaging area includes at least the face areas at both ends among the detected plurality of face areas.

15. The image processing device according to claim 1 ,
wherein the imaging device includes a first imaging device that photographs an image from a same direction as a facing direction of the display device and a second imaging device that photographs an image from an opposite direction to the facing direction of the display device and is configured such that the user can select an imaging device to be used among the first imaging device and the second imaging device, and
the display manner of the guide changes according to the selected imaging device to be used.

16. The image processing device according to claim 3 ,
wherein the imaging device includes a first imaging device that photographs an image from a same direction as a facing direction of the display device and a second imaging device that photographs an image from an opposite direction to the facing direction of the display device and is configured such that the user can select an imaging device to be used among the first imaging device and the second imaging device, and
the total imaging area includes an imaging area of the first imaging device and an imaging area of the second imaging device.

17. The image processing device according to claim 1 , wherein
the processor is further configured to:
detect a face area of a person displayed on the display device, and
set a display location of the guide based on the face area detected by the processor.

18. The image processing device according to claim 1 , wherein the processor is further configured to display the guide near a first imaging device that photographs an image from the same direction as a facing direction of the display device.

19. The image processing device according to claim 1 , wherein the processor is further configured to display the guide at a higher visibility than the photographing subject until the imaging device is rotated by a predetermined degree of rotation.

20. The image processing device according to claim 1 ,
wherein the processor is fu

Assignees

• Morpho Inc

Inventors

• Hiraga Masaki
• HIRAI Shun
• Miura Takeshi
• Ono Ryo

Classifications

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/61

  based on recognised objects · CPC title

• H04N23/635

  Region indicators; Field of view indicators · CPC title

• H04N23/698 Primary

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• H04N23/631

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title