Program, method, and information terminal device

Patent metadata

Field	Value
Publication number	US-12357910-B2
Application number	US-202017606633-A
Country	US
Kind code	B2
Filing date	Jun 22, 2020
Priority date	Apr 26, 2019
Publication date	Jul 15, 2025
Grant date	Jul 15, 2025

Abstract

An image of an augmented reality space, in which a character is arranged with respect to an acquired image acquired by the imaging sensor, is displayed on the display unit, and the behavior of the character in the augmented reality space is produced on the basis of data transmitted from an external source, wherein the data is motion data and sound data that are input by a performer who plays live as the character.

First claim

The invention claimed is:

1. A computer readable storage medium containing a program to be executed by at least one information terminal device of a plurality of information terminal devices in a system that includes: a game play terminal configured to be operated by a performer who carries on a game; and the plurality of information terminal devices configured to be operated by a plurality of users who support a progress of the game, each information terminal device comprising a processor, a memory, an input unit, a display unit, and an imaging sensor,
the program, when executed by the processor, causing the processor to execute steps of:
arranging a virtual camera to acquire an image of a character that is arranged in a virtual space for augmented reality defined for the at least one information terminal device, wherein the character is an object that behaves as an avatar of the performer;
generating an image of an augmented reality space by superimposing, on an acquired image acquired by the imaging sensor, a field-of-view image including an image of the character acquired by the virtual camera;
displaying, on the display unit, the generated image of an augmented reality space;
producing a behavior of the character on a basis of transmission data transmitted from the game play terminal, wherein the transmission data is motion data and sound data that are input by the performer who plays live as the character;
displaying on the display unit an image in which the character is arranged in a similar virtual space to a virtual space defined in the game play terminal; and
switching, depending on an operation input by a user, the image displayed on the display unit between the image of the augmented reality space and the image in which the character is arranged in the similar virtual space,
wherein the motion data and the sound data that are input by the performer are commonly transmitted to the plurality of information terminal devices used by a plurality of users, and
wherein the behavior of the character is common among the plurality of information terminal devices, while images of the augmented reality space where the character is arranged, in the plurality of information terminal devices, are different depending on acquired images acquired by imaging sensors of the plurality of information terminal devices, respectively.

2. The computer readable storage medium according to claim 1 , wherein the program, when executed by the processor, causes the processor to execute a step of adjusting a size of the character based on an operation input by the user.

3. The computer readable storage medium according to claim 1 , wherein:
the transmission data includes the motion data and the sound data that are input by the performer who carries on the game live; and
the step of producing the behavior includes producing a behavior of the character who carries on the game based on the transmission data.

4. The computer readable storage medium according to claim 3 , wherein the game progresses based on the behavior of the character and an operation input by the user on the at least one information terminal device.

5. The computer readable storage medium according to claim 4 , wherein:
the transmission data is commonly transmitted to the plurality of information terminal devices used by a plurality of users; and
the behavior of the character is common among the plurality of information terminal devices, while results of the game in the plurality of information terminal devices are different depending on operations input by users on the plurality of information terminal devices, respectively.

6. The computer readable storage medium according to claim 1 , wherein the program, when executed by the processor, causes the processor to execute a step of displaying on the display unit a UI (User Interface) allowing a user to perform an action on the performer.

7. The computer readable storage medium according to claim 6 , wherein:
the transmission data includes the motion data and the sound data that are input by the performer who carries on the game;
the step of producing the behavior includes producing a behavior of the character who carries on the game in the augmented reality space, based on the transmission data; and
the UI includes a UI for performing the action in accordance with progress of the game.

8. The computer readable storage medium according to claim 1 , wherein the step of displaying on the display unit the image of the augmented reality space includes:
displaying on the display unit an image in which a virtual space containing the character as a whole is arranged with respect to the acquired image, when the transmission data is a first type of content data; and
displaying on the display unit an image in which a part of the virtual space containing the character is arranged with respect to the acquired image, when the transmission data is a second type of content data.

9. A method to be executed by at least one information terminal device of a plurality of information terminal devices in a system that includes: a game play terminal configured to be operated by a performer who carries on a game; and the plurality of information terminal devices configured to be operated by a plurality of users who support a progress of the game, each information terminal device comprising a processor, a memory, an input unit, a display unit, and an imaging sensor, the method comprising steps of:
arranging a virtual camera to acquire an image of a character that is arranged in a virtual space for augmented reality defined for the at least one information terminal device, wherein the character is an object that behaves as an avatar of the performer;
generating an image of an augmented reality space by superimposing, on an acquired image acquired by the imaging sensor, a field-of-view image including an image of the character acquired by the virtual camera;
displaying, on the display unit, the generated image of an augmented reality space;
producing a behavior of the character on a basis of transmission data transmitted from the game play terminal, wherein the transmission data is motion data and sound data that are input by the performer who plays live as the character;
displaying on the display unit an image in which the character is arranged in a similar virtual space to a virtual space defined in the game play terminal; and
switching, depending on an operation input by a user, the image displayed on the display unit between the image of the augmented reality space and the image in which the character is arranged in the similar virtual space,
wherein the motion data and the sound data that are input by the performer are commonly transmitted to the plurality of information terminal devices used by a plurality of users, and
wherein the behavior of the character is common among the plurality of information terminal devices, while images of the augmented reality space where the character is arranged, in the plurality of information terminal devices, are different depending on acquired images acquired by imaging sensors of the plurality of information terminal devices, respectively.

10. The method according to claim 9 , further comprising a step of adjusting a size of the character based on an operation input by the user.

11. The method according to claim 9 , wherein:
the transmission data includes the motion data and the sound data that are input by the performer who carries on the game live; and
the step of producing the behavior includes producing a behavior of the character who carries on the game based on the transmission data.

12. The method according to claim 11 , wherein t

Assignees

• Colopl Inc

Inventors

• Fukushige Junya

Classifications

• G06T19/006

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T13/40

  of characters, e.g. humans, animals or virtual beings · CPC title

• A63F13/53

  involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game · CPC title

• A63F13/525

  Changing parameters of virtual cameras · CPC title

• A63F2300/8082

  Virtual reality · CPC title