Device and method for processing high-definition 360-degree VR image

Patent metadata

Field	Value
Publication number	US-10848837-B2
Application number	US-201816492289-A
Country	US
Kind code	B2
Filing date	Apr 3, 2018
Priority date	Apr 3, 2017
Publication date	Nov 24, 2020
Grant date	Nov 24, 2020

Abstract

Disclosed is an apparatus and method of providing a high quality 360-degree VR image. A method of decoding a 360-degree VR image according to the present disclosure includes: receiving a bit stream including 360-degree VR image information; decoding information related to a 360-degree VR service from the bitstream; detecting a region of interest based on the information related to the 360-degree VR service; and providing to a user a 360-degree VR image for the region of interest.

First claim

The invention claimed is:

1. A method of decoding a 360-degree image, the method comprising:
receiving a bitstream including 360-degree virtual reality (VR) image information;
obtaining respectively a first 360-degree VR image having a first resolution and a second 360-degree VR image having a second resolution higher than the first resolution;
decoding first signaling information and second signaling information related to a 360-degree VR service from the bitstream;
detecting a region of interest based on the first signaling information and second signaling information; and
providing to a user an image for the region of interest from at least one of the first 360-degree VR image and the second 360-degree VR image, based on whether the region of interest is being changed or the region of interest is selected,
wherein the first signaling information is applied to a program or to a service level, and
wherein the second signaling information is applied to a stream level of each of a plurality of sub-regions generated by dividing the second 360-degree VR image.

2. The method of claim 1 , wherein the first signaling information includes projection type information used to perform mapping the first 360-degree image and the second 360-degree image, and division information indicating a number of the plurality of sub-regions.

3. The method of claim 1 , wherein the second signaling information includes first 360-degree image selection information indicating whether or not to provide the first 360-degree image to the user, and identification information about at least one of the plurality of sub-regions to be provided firstly to the user.

4. The method of claim 1 , wherein a size of the image for the region of interest is re-adjusted to be suitable for providing to the user.

5. The method of claim 1 , when the region of interest is being changed, wherein the image for the region of interest is provided to the user by performing mapping the first 360-degree image and at least one of the plurality of sub-regions corresponding to the region of interest.

6. The method of claim 1 , when the region of interest is selected, wherein the image for the region of interest is provided to the user from at least one sub-region corresponding to the region of interest among the plurality of sub-regions, without the first 360-degree VR image.

7. An apparatus for decoding a 360-degree image, the apparatus comprising:
a receiving module receiving a bitstream including 360-degree VR image information;
a buffer module obtaining respectively a first 360-degree VR image having a first resolution and a second 360-degree VR image having a second resolution higher than the first resolution;
a viewport control module extracting an image for a region of interest from a plurality of sub-regions of the second 360-degree image, based on at least one of motion information of a user and an input of the user, the plurality of sub-regions are generated by dividing the second 360-degree image;
a decoder module decoding signaling information related to a 360-degree VR service, the first 360-degree image and the image for the region of interest;
an RoI mapping module re-adjusting a size of the image for the region of interest, based on a resolution information supported by a terminal of the user; and
a 360-degree VR mapping module performing mapping the image for the region of interest and the first 360-degree VR image, based on whether the region of interest is being changed or the region of interest is selected.

8. The apparatus of claim 7 , the signaling information includes first signaling information being applied to a program or to a service level, and second signaling information being applied to a stream level of each of a plurality of sub-regions generated by dividing the second 360-degree VR image.

9. The apparatus of claim 8 , wherein the first signaling information includes projection type information used to perform mapping the first 360-degree image and the second 360-degree image, and division information indicating a number of the plurality of sub-regions.

10. The apparatus of claim 8 , wherein the second signaling information includes first 360-degree image selection information indicating whether or not to provide the first 360-degree image to the user.

11. The apparatus of claim 10 , when the first 360-degree image is not provided to the user, wherein the second signaling information further includes identification information about at least one of the plurality of sub-regions to be provided firstly to the user.

12. The apparatus of claim 8 , wherein the viewport control module extracts the image for the region of interest as at least one of a plurality of sub-regions of the second 360-degree VR image.

13. The method of claim 1 , wherein the obtained first 360-degree VR image and the obtained second 360-degree VR image are synchronized based on at least one of a decoding time stamp (DTS) and a presentation time stamp (PTS).

14. The apparatus of claim 7 , wherein the buffer module synchronizes the obtained first 360-degree image and the obtained second 360-degree image, based on at least one of a decoding time stamp (DTS) and a presentation time stamp (PTS).

15. The apparatus of claim 7 , when the region of interest is selected, wherein the 360-degree VR mapping module outputs the image for the region of interest without performing mapping with the first 360-degree VR image.

16. A method of encoding a 360-degree image, the method comprising:
receiving 360-degree VR image information related to a first 360-degree VR image having first resolution and a second 360-degree VR image having second resolution higher than the first resolution;
recognizing a region of interest of a user; and
encoding the first 360-degree VR image, the second 360-degree VR image, first signaling information used to distinguish the first 360-degree VR image and the second 360-degree VR image, and second signaling information used to provide an image for the region of interest,
wherein the second 360-degree VR image is divided into a plurality of sub-regions, and at least one of the plurality of sub-regions for the region of interest is encoded and output through one or more channels,
wherein the first signaling information is applied to a program or to a service level, and
wherein the second signaling information is applied to a stream level of each of the plurality of sub-regions.

17. The method of claim 16 ,
wherein the first signaling information includes projection type information used to perform mapping the first 360-degree image and the second 360-degree image, and division information indicating a number of the plurality of sub-regions, and
wherein the second signaling information includes first 360-degree image selection information indicating whether or not to provide the first 360-degree image to the user, and identification information about at least one of the plurality of sub-regions to be provided firstly to the user.

18. The method of claim 16 , when the first 360-degree image is not provided to the user, wherein the second signaling information further includes identification information about at least one of the plurality of sub-regions to be provided firstly to the user.

19. An apparatus for encoding a 360-degree image, wherein the apparatus receives 360-degree VR image information related to a first 360-degree VR image having first resolution and a second 360-degree VR image having second resolution higher than the first resolution, recognizes a region of inte

Assignees

• Electronics & Telecommunications Res Inst

Inventors

• Yun Kug Jin
• Jeong Jun Young

Classifications

• H04N19/162

  User input · CPC title

• H04N21/816 Primary

  involving special video data, e.g 3D video · CPC title

• H04N21/21805 Primary

  enabling multiple viewpoints, e.g. using a plurality of cameras · CPC title

• H04N13/161

  Encoding, multiplexing or demultiplexing different image signal components (for multi-view video sequence encoding H04N19/597) · CPC title

• H04N21/2362

  Generation or processing of Service Information [SI] · CPC title