Virtual space interface device, client terminal, computer readable non-transitory storage medium storing program, and virtual space interface control method

Patent metadata

Field	Value
Publication number	US-2025348150-A1
Application number	US-202519090027-A
Country	US
Kind code	A1
Filing date	Mar 25, 2025
Priority date	Sep 27, 2022
Publication date	Nov 13, 2025
Grant date	—

Abstract

A virtual space interface device generates display data for causing a client terminal to display an image showing a situation in a virtual space, generates sound data for outputting a user-uttered sound picked up by the terminal into the virtual space, and generates sound data for causing the terminal to output a sound in the virtual space. The display data and the sound data are controlled on the basis of a gesture of the user and a positional relationship between the user and the terminal. A control target differs in accordance with a part of a face area where the user positions the user's hands.

First claim

1 . A virtual space interface device provided in a virtual space providing system having at least a client terminal used by a user,
wherein the client terminal includes a display device configured to display an image showing a situation in the virtual space, a sound output device configured to output a sound in the virtual space, a sound pickup device configured to pick up a sound uttered by the user, and a photographing device configured to capture a facial image of the user, wherein the virtual space interface device includes a display data generating unit configured to generate display data for causing the display device of the client terminal to display an image showing a situation in the virtual space and a sound data generating unit configured to generate sound data for causing the sound output device of the client terminal to output a sound in the virtual space, wherein the sound data generating unit generates sound data for outputting the user-uttered sound picked up by the sound pickup device of the client terminal into the virtual space, wherein the display data generating unit and the sound data generating unit control at least one item of the display data for causing the display device of the client terminal to display the image showing the situation in the virtual space, the sound data for causing the sound output device of the client terminal to output the sound in the virtual space, and the sound data for outputting the sound uttered by the user into the virtual space, as a control target, on the basis of a gesture of positioning the user's hands at an area of the user's face photographed by the photographing device of the client terminal and a positional relationship between the photographing device of the client terminal and the user's face, and wherein the display data generating unit and the sound data generating unit differentiate the control target in accordance with a part of the face area where the user positions the user's hands.

2 . The virtual space interface device according to claim 1 ,
wherein the display data generating unit controls at least one of enlargement and reduction of the image showing the situation in the virtual space displayed by the display device of the client terminal on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hands over the user's eyes and a distance between the photographing device of the client terminal and the user's face, wherein the sound data generating unit controls a volume of the sound in the virtual space output by the sound output device of the client terminal on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hands at the user's ears and the distance between the photographing device of the client terminal and the user's face, and wherein the sound data generating unit controls a volume of the user-uttered sound picked up by the sound pickup device of the client terminal and output into the virtual space on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hand at the user's mouth and the distance between the photographing device of the client terminal and the user's face.

3 . The virtual space interface device according to claim 1 ,
wherein the display data generating unit controls a position corresponding to the image displayed by the display device of the client terminal as a position in the virtual space on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hands over the user's eyes and an orientation of the user's face relative to the photographing device of the client terminal, wherein the sound data generating unit controls a direction of arrival of the sound from the virtual space output by the sound output device of the client terminal on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hands at the user's ears and the orientation of the user's face relative to the photographing device of the client terminal, and wherein the sound data generating unit controls a direction in which the sound uttered by the user is output to the virtual space on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hand at the user's mouth and the orientation of the user's face relative to the photographing device of the client terminal.

4 . A virtual space interface device provided in a virtual space providing system having at least a client terminal used by a user,
wherein the client terminal includes a sound output device configured to output a sound in the virtual space, a sound pickup device configured to pick up a sound uttered by the user, and a photographing device configured to capture a facial image of the user, wherein the virtual space interface device includes a sound data generating unit configured to generate sound data for causing the sound output device of the client terminal to output a sound in the virtual space, wherein the sound data generating unit generates sound data for outputting the user-uttered sound picked up by the sound pickup device of the client terminal into the virtual space, and wherein the sound data generating unit controls at least one item of the sound data for causing the sound output device of the client terminal to output the sound in the virtual space and the sound data for outputting the sound uttered by the user into the virtual space, as a control target, on the basis of a gesture of positioning the user's hands at an area of the user's face photographed by the photographing device of the client terminal and a positional relationship between the photographing device of the client terminal and the user's face and differentiates the control target in accordance with a part of the face area where the user positions the user's hands.

5 . The virtual space interface device according to claim 4 , wherein the sound data generating unit controls a volume of the sound in the virtual space output by the sound output device of the client terminal on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hands at the user's ears and a distance between the photographing device of the client terminal and the user's face.

6 . The virtual space interface device according to claim 4 , wherein the sound data generating unit controls a volume of the user-uttered sound picked up by the sound pickup device of the client terminal and output into the virtual space on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hand at the user's mouth and the distance between the photographing device of the client terminal and the user's face.

7 . The virtual space interface device according to claim 4 , wherein the sound data generating unit controls a direction of arrival of the sound from the virtual space output by the sound output device of the client terminal on the basis of an action of the user, who is photographed by the photographing device of the client terminal, placing the user's hands at the user's ears and the orientation of the user's face relative to the photographing device of the client terminal.

8 . The virtual space interface device according to claim 4 , wherein the sound data generating unit controls a direction in which the sound uttered by the user is output to the virtual space on the basis of an action of the user, who is photographed by the phot

Assignees

• Jvckenwood Corp

Inventors

• Yamamoto Shunsuke
• Takiwaki Aiko
• Fujishima Moe
• Matsumoto Yuichi
• Kim Hyungjun
• Hayashishita Yu
• Sato Yukako
• Seki Kazuya
• Shiga Minoru

Classifications

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title

• G06F3/165

  Management of the audio stream, e.g. setting of volume, audio stream path · CPC title

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title