Apparatus and method for acquiring in-game 360 VR image by using plurality of virtual cameras

Patent metadata

Field	Value
Publication number	US-11951394-B2
Application number	US-202117235528-A
Country	US
Kind code	B2
Filing date	Apr 20, 2021
Priority date	Oct 22, 2018
Publication date	Apr 9, 2024
Grant date	Apr 9, 2024

Abstract

An image acquisition apparatus for acquiring an in-game 360 virtual reality (VR) image by using a plurality of virtual cameras includes a virtual camera group taking in-game images by using the plurality of virtual cameras. The image acquisition apparatus may also include a renderer generating textures for the taken images, generating a panoramic image by performing an equirectangular projection (ERP) mapping of the generated textures, and encoding the generated panoramic image. The image acquisition apparatus may further include an image generator generating a 360 VR image by combining audio information with the encoded panoramic image.

First claim

What is claimed is:

1. An in-game 360 virtual reality (VR) image acquisition apparatus using a plurality of virtual cameras, the apparatus comprising:
a virtual camera group configured to take in-game images by using the plurality of virtual cameras;
a renderer configured to generate textures for the taken images, generate a panoramic image by performing an equirectangular projection (ERP) mapping of the generated textures, and encode the generated panoramic image; and
an image generator configured to generate a 360 VR image by combining audio information with the encoded panoramic image, and further configured to video-stream the generated 360 VR image and output the video-streamed 360 VR image to an outside of the apparatus,
wherein in the virtual camera group, omni-direction from a reference point for taking the images is divided into N (N is a natural number greater than 4), and a pair of left and right virtual cameras are arranged in each of N directions.

2. The apparatus of claim 1 , wherein in the virtual camera group, omni-direction from a reference point for taking the images is divided into 4, the pair of left and right virtual cameras are arranged in each direction facing front, back, left, and right from a reference point for taking the images, and wherein the pair of left and right virtual cameras are arranged in each direction facing top and bottom of the front.

3. The apparatus of claim 2 , wherein the respective virtual cameras of the virtual camera group are arranged to have an angle of π/2.

4. The apparatus of claim 1 , top and bottom virtual cameras are arranged in each of the N directions.

5. The apparatus of claim 1 , wherein the respective virtual cameras of the virtual camera group are arranged to have an angle of 2π/N.

6. The apparatus of claim 1 , wherein the renderer includes:
a texture generator configured to capture the images and generate textures for the respective images;
a general-purpose computing on graphics processing unit (GPGPU) ERP engine configured to perform ERP mapping of the generated textures by using pixel mapping data indicating a location of a pixel to be mapped, and generate the panoramic image through the ERP mapping; and
an encoder configured to encode the generated panoramic image for each frame.

7. The apparatus of claim 6 , wherein the GPGPU ERP engine is configured to perform respective steps in a parallel structure of a pipeline scheme.

8. The apparatus of claim 6 , wherein the GPGPU ERP engine is configured to generate in advance the pixel mapping data in an initialization process.

9. The apparatus of claim 6 , wherein the GPGPU ERP engine is configured to get a writer from a circular buffer disposed between the GPGPU ERP engine and the encoder to perform the ERP mapping.

10. An image acquisition method to be performed in an in-game 360 virtual reality (VR) image acquisition apparatus using a plurality of virtual cameras, the method, comprising:
at an image acquisition apparatus, taking in-game images by using a virtual camera group comprising a plurality of virtual cameras;
at the image acquisition apparatus, generating textures for the taken images, generating a panoramic image by performing an equirectangular projection (ERP) mapping of the generated textures, and encoding the generated panoramic image;
at the image acquisition apparatus, generating a 360 virtual reality (VR) image by combining audio information with the encoded panoramic image;
at the image acquisition apparatus, video-streaming the generated 360 VR image; and
at the image acquisition apparatus, outputting the video-streamed 360 VR image to an outside of the image acquisition apparatus,
wherein in the virtual camera group, omni-direction from a reference point for taking the images is divided into N (N is a natural number greater than 4), and a pair of left and right virtual cameras are arranged in each of N directions.

Assignees

• Korea Electronics Technology

Inventors

• Jang Junhwan
• Park Woochool
• Yang Jinwook
• Yoon Sangpil
• Choi Minsu
• Lee Junsuk
• Song Suho
• Koo Bonjae

Classifications

• A63F13/5252 Primary

  using two or more virtual cameras concurrently or sequentially, e.g. automatically switching between fixed virtual cameras when a character changes room or displaying a rear-mirror view in a car-driving game · CPC title

• H04N23/698

  for achieving an enlarged field of view, e.g. panoramic image capture · CPC title

• G06T15/04 Primary

  Texture mapping · CPC title

• A63F13/86

  Watching games played by other players · CPC title

• H04N13/282

  for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems · CPC title