Computer readable medium storing image processing program of synthesizing images

Patent metadata

Field	Value
Publication number	US-9737814-B2
Application number	US-201113231344-A
Country	US
Kind code	B2
Filing date	Sep 13, 2011
Priority date	Oct 15, 2010
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

A real world image captured by a real camera such as an outside right imaging unit 23 b is acquired, a synthesized image is generated by synthesizing the acquired real world image and a virtual image depicting a first virtual object such as an enemy object EO, in such a manner that the first virtual object such as an enemy object EO appears to be present behind the real world image, and the synthesized image thus generated is displayed on a display device.

First claim

What is claimed is:

1. A non-transitory computer readable medium storing an image processing program that is executed by a computer which displays a synthesized image of a real world image and a virtual image on a display device, the image processing program causing the computer to perform functionality comprising:
acquiring a real world image captured by a real camera;
defining first and second virtual regions in a virtual image based on a boundary surface adjoining the first and second virtual regions, the first virtual region located inside the boundary surface and the second virtual region located outside the boundary surface;
generating a synthesized image by synthesizing the real world image and the virtual image depicting a first virtual object, in such a manner that the first virtual object appears to be present behind the real world image when the first virtual object is in the second virtual region; and
displaying the synthesized image on the display device, wherein a portion of the first virtual object, appearing to be present behind the real world image, is depicted as a silhouette of the first virtual object.

2. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein the synthesized image is generated by performing synthesis in such a manner that the real world image is displayed preferentially with respect to the virtual image depicting the first virtual object in the generating the synthesized image.

3. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein the synthesized image is generated in such a manner that a second virtual object which is present in the first virtual region that is different from the second virtual region appears to be present in front of the real world image in the generating the synthesized image.

4. The non-transitory computer readable medium storing the image processing program according to claim 3 , wherein the synthesized image is generated by performing synthesis in such a manner that the virtual image depicting the second virtual object is displayed preferentially with respect to the real world image in the generating the synthesized image.

5. The non-transitory computer readable medium storing the image processing program according to claim 3 , wherein the second virtual object is synthesized by using the real world image as a background in the generating the synthesized image; and
the synthesized image is generated in such a manner that the first virtual object appears to be present behind the background in the generating the synthesized image.

6. The non-transitory computer readable medium storing the image processing program according to claim 3 , wherein a first region of a predetermined virtual space is set as the first virtual region in the generating the synthesized image; and
a second region of the virtual space is set as the second virtual region in the generating the synthesized image.

7. The non-transitory computer readable medium storing the image processing program according to claim 6 , wherein adjacent regions are set as the first region and the second region in the generating the synthesized image.

8. The non-transitory computer readable medium storing the image processing program according to claim 6 , wherein
the first region includes a position of a virtual camera; and
the second region is a region more distant from the position of the virtual camera than the first region.

9. The non-transitory computer readable medium storing the image processing program according to claim 6 , wherein depth judgment is performed, with reference to the virtual camera, in respect of the boundary surface between the first region and the second region of the virtual space, and the first virtual object and the second virtual object, and on the basis of this depth judgment, a virtual object that is judged to be nearer to the virtual camera than the boundary surface with respect to the real world image is preferentially synthesized, and the real world image with respect to a virtual object that is judged to be further from the virtual camera than the boundary surface is preferentially synthesized in the generating the synthesized image.

10. The non-transitory computer readable medium storing the image processing program according to claim 6 , wherein the synthesized image is generated by applying the real world image to a screen object which encompasses a range of view of the virtual camera and is disposed at a boundary position between the first region and the second region, and by imaging the first virtual object, the second virtual object and the screen object by the virtual camera in the generating the synthesized image.

11. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein the synthesized image is generated in such a manner that the real world image occupies the whole area of the synthesized image in the generating the synthesized image.

12. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein the computer is further caused to perform functionality comprising repeatedly acquiring at least an imaging direction of the real camera;
wherein the real world image is acquired repeatedly from the real camera in the acquiring the real world image; and
in the generating the synthesized image, the virtual image is generated by setting a current imaging direction of the virtual camera in accordance with a current imaging direction of the real camera acquired in the acquiring the imaging direction of the real camera and the synthesized image is generated by using current real world images acquired repeatedly in the acquiring the real world image.

13. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein the silhouette has an outline corresponding to an orientation of the first virtual object.

14. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein the silhouette of the first virtual object is viewed as being unobstructed when the virtual object appears to be present behind the real world image.

15. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein the synthesized image is generated in such a manner that the first virtual object appears to be present behind a whole of the real world image.

16. The non-transitory computer readable medium storing the image processing program according to claim 1 , wherein a virtual camera is positioned within the first virtual region inside the boundary surface.

17. A non-transitory computer readable medium storing an image processing program that is executed by a computer which displays a synthesized image of a real world image and a virtual image on a display device, the image processing program causing the computer to perform functionality comprising:
acquiring a real world image captured by a real camera;
defining first and second virtual regions in a virtual image based on a boundary surface adjoining the first and second virtual regions, the first virtual region located inside the boundary surface and the second virtual region located outside the boundary surface;
generating a synthesized image by synthesizing the real world image and the virtual image depicting a first virtual object, in such a manner that the first virtual object appears to be present behind the real world image when the first vir

Assignees

• Suzuki Toshiaki
• Kawase Shigefumi
• Nintendo Co Ltd
• Hal Laboratory Inc

Inventors

• Suzuki Toshiaki
• Kawase Shigefumi

Classifications

• A63F13/655 Primary

  by importing photos, e.g. of the player · CPC title

• A63F13/213 Primary

  comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) · CPC title

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• A63F2300/8076

  Shooting · CPC title

• A63F13/52

  involving aspects of the displayed game scene · CPC title