Method for processing omni-directional image using padding area and apparatus supporting the same

Patent metadata

Field	Value
Publication number	US-11017598-B2
Application number	US-201716342392-A
Country	US
Kind code	B2
Filing date	Oct 18, 2017
Priority date	Oct 18, 2016
Publication date	May 25, 2021
Grant date	May 25, 2021

Abstract

An image processing apparatus is disclosed. The image processing apparatus comprises: a receiver configured to receive an input frame including a plurality of image regions, corresponding to a plurality of faces of a three-dimensional polyhedron, and metadata; and a processor configured to render an output frame including at least one a part of the input frame, on the basis of padding information included in the metadata. Here, the processor may be configured to identify, on the basis of the padding information, a padding region included in at least one of the plurality of image regions, and render, on the basis of the determined padding region, a boundary between at least one face of the plurality of faces and another face of the plurality of faces adjoining the at least one face of the polyhedron.

First claim

The invention claimed is:

1. An apparatus for processing an image, the apparatus comprising:
a communication circuit configured to receive an input frame including a plurality of image areas corresponding to respective faces of a polyhedron and metadata; and
a processor configured to render an output frame including at least part of the input frame, based on padding information included in the metadata,
wherein the processor is configured to:
determine a padding area included in at least one of the plurality of image areas, based on the padding information, and
render a boundary between at least one face of the polyhedron and another face of the polyhedron concatenated with the at least one face, based on the determined padding area,

wherein the padding information includes at least one piece of size information of a padding area included in at least one of the plurality of image areas and resolution information of at least one of the plurality of image areas, and
wherein, when a resolution of the at least one face of the polyhedron is higher than a resolution of the another face different from the at least one face of the polyhedron, the processor sets a size of an area used for rendering a boundary among padding areas included in the at least one face of the polyhedron to be larger than a size of an area used for rendering a boundary among the padding areas included in the another face different from the at least one face of the polyhedron and renders the boundary among the padding areas included in the at least one face.

2. The apparatus of claim 1 ,
wherein the metadata further includes arrangement attribute information indicating a mapping relation between the respective faces of the polyhedron and each of the plurality of image areas, and
wherein the processor is configured to:
determine image areas corresponding to a least one face of the polyhedron among the plurality of image areas and another face of the polyhedron concatenated with the at least one face based on the arrangement attribute information, and
render the at least one face of the polyhedron and the another face of the polyhedron concatenated with the at least one face of the polyhedron based on the determined image areas.

3. An apparatus for processing an image, the apparatus comprising:
a memory configured to store an input frame and metadata; and
a processor configured to:
generate an output frame including at least one of a plurality of image areas to which images of respective faces of a polyhedron are mapped, based on the input frame and the metadata and control the memory to store the generated output frame,
generate at least one padding area, based on padding information included in the metadata, and
insert the at least one generated padding area into at least one of the plurality of image areas to generate the output frame,

wherein the padding information includes at least one piece of size information of a padding area included in at least one of the plurality of image areas and resolution information of at least one of the plurality of image areas, and
wherein, when a resolution of at least one face of the polyhedron is higher than a resolution of another face different from the at least one face of the polyhedron, the processor sets a size of an area used for rendering a boundary among padding areas included in the at least one face of the polyhedron to be larger than a size of an area used for rendering a boundary among the padding areas included in the another face different from the at least one face of the polyhedron and renders the boundary among the padding areas included in the at least one face.

4. The apparatus of claim 3 , wherein the metadata further includes arrangement attribute information indicating a mapping relation between the respective faces of the polyhedron and each of the plurality of image areas, and the processor maps the images of the respective faces of the polyhedron to the plurality of image areas, based on the arrangement attribute information.

5. A method of processing an image, the method comprising:
receiving an input frame including a plurality of image areas corresponding to respective faces of a polyhedron and metadata; and
rendering an output frame including at least part of the input frame, based on padding information included in the metadata,
wherein the rendering of the output frame comprises:
determining a padding area included in at least one of the plurality of image areas, based on the padding information; and
rendering a boundary between at least one face of the polyhedron and another face of the polyhedron concatenated with the at least one face, based on the determined padding area,

wherein the padding information includes at least one piece of size information of a padding area included in at least one of the plurality of image areas and resolution information of at least one of the plurality of image areas, and
wherein the rendering of the output frame comprises:
when a resolution of the at least one face of the polyhedron is higher than a resolution of the another face different from the at least one face of the polyhedron, setting a size of an area used for rendering a boundary among padding areas included in the at least one face of the polyhedron to be larger than a size of an area used for rendering a boundary among the padding areas included in the another face different from the at least one face of the polyhedron and rendering the boundary among the padding areas included in the at least one face of the polyhedron.

6. The method of claim 5 ,
wherein the metadata further includes arrangement attribute information indicating a mapping relation between the respective faces of the polyhedron and each of the plurality of image areas, and
wherein the rendering of the output frame comprises:
determining image areas corresponding to a least one face of the polyhedron among the plurality of image areas and another face of the polyhedron concatenated with the at least one face based on the arrangement attribute information; and
rendering the at least one face of the polyhedron and the another face of the polyhedron concatenated with the at least one face of the polyhedron based on the determined image areas.

7. A method of processing an image, the method comprising:
receiving an input frame and metadata; and
generating an output frame including at least one of a plurality of image areas to which images of respective faces of a polyhedron are mapped, based on the input frame and the metadata,
wherein the generating comprises generating at least one padding area, based on padding information included in the metadata and inserting the at least one generated padding area into at least one of the plurality of image areas to generate the output frame,
wherein the padding information includes at least one piece of size information of a padding area included in at least one of the plurality of image areas and resolution information of at least one of the plurality of image areas, and
wherein the generating of the at least one padding area comprises:
when a resolution of the at least one face of the polyhedron is higher than a resolution of another face different from the at least one face of the polyhedron, setting a size of an area used for rendering a boundary among padding areas included in the at least one face of the polyhedron to be larger than a size of an area used for rendering a boundary among the padding areas included in the another face different from the at least one face of the polyhedron and rendering the boundary among the padding areas included in the at least one face of the polyhedron.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Choi Byung-Po
• Yu In-Su
• Lim Jin-Ho
• Jung Il-Hoe
• Sena David Bernardino Martins
• Garnier Frederic
• Kim Yoon-Joo
• Lee Jung-Eun

Classifications

• G06T2207/20068

  Projection on vertical or horizontal image axis · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T2207/20016

  Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform · CPC title

• G06T15/20 Primary

  Perspective computation · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title