Method and apparatus for image colorization

Patent metadata

Field	Value
Publication number	US-2016307306-A1
Application number	US-201615095027-A
Country	US
Kind code	A1
Filing date	Apr 9, 2016
Priority date	Apr 20, 2015
Publication date	Oct 20, 2016
Grant date	—

Abstract

A method and an apparatus for image colorization. A first image obtaining unit obtains an original image. A second image obtaining unit obtains an auxiliary image comprising information about target colors for pixels of the original image. A Guided Image Filter then filters the auxiliary image using the original image as a Guiding Image to generate an output image.

First claim

What is claimed, is:

1 . A method for image colorization, the method comprising:
obtaining an original image; obtaining an auxiliary image, wherein the auxiliary image comprises information about target colors for pixels of the original image; and filtering the auxiliary image with a Guided Image Filter using the original image as a Guiding Image to generate an output image.

2 . The method according to claim 1 , wherein the auxiliary image comprises information about a first type of pixels, for which a target color is specified, and a second type of pixels, for which no target color is specified.

3 . The method according to claim 2 , wherein for the first type of pixels, the target color of a pixel is one of the color of the pixel in the original image and a new color.

4 . The method according to claim 2 , wherein the Guided Image Filter is a Confidence-Aware Guided Image Filter.

5 . The method according to claim 4 , wherein a small confidence of ‘0’ or a value in the vicinity of ‘0’ is assigned to a pixel without a specified target color and a larger confidence of ‘1’ or a value in the vicinity of ‘1’ is assigned to a pixel with a specified target color.

6 . The method according to claim 5 , wherein the larger confidence is assigned to a pixel having the color of the pixel in the original image as a specified target color if and only if a distance of the pixel to another pixel without a specified target color is smaller than a distance threshold, and wherein otherwise the smaller confidence is assigned to the pixel.

7 . The method according to claim 2 , further comprising generating a refined auxiliary image by:
generating a trimap from the auxiliary image; generating an alpha matte from the original image and the trimap; and assigning target colors to pixels of the auxiliary image without a specified target color based on the alpha matte.

8 . The method according to claim 7 , wherein assigning target colors to pixels of the auxiliary image without a specified target color based on the alpha matte comprises:
assigning the color of the pixel in the original image as a specified target color to a pixel if an alpha value for the pixel is smaller than a lower alpha threshold in the vicinity of ‘0’; and assigning the new color as a specified target color to a pixel if the alpha value for the pixel is larger than an upper alpha threshold in the vicinity of ‘1’.

9 . A non-transitory computer readable storage medium having stored therein instructions enabling image colorization, which when executed by a computer, cause the computer to:
obtain an original image; obtain an auxiliary image, wherein the auxiliary image comprises information about target colors for pixels of the original image; and filter the auxiliary image with a Guided Image Filter using the original image as a Guiding Image to generate an output image.

10 . An apparatus for image colorization, the apparatus comprising:
a first image obtaining unit configured to obtain an original image; a second image obtaining unit configured to obtain an auxiliary image, wherein the auxiliary image comprises information about target colors for pixels of the original image; and a Guided Image Filter configured to filter the auxiliary image using the original image as a Guiding Image to generate an output image.

11 . An apparatus for image colorization, the apparatus comprising a processing device and a memory device having stored therein instructions, which, when executed by the processing device, cause the apparatus to:
obtain an original image; obtain an auxiliary image, wherein the auxiliary image comprises information about target colors for pixels of the original image; and filter the auxiliary image with a Guided Image Filter using the original image as a Guiding Image to generate an output image.

12 . A method for generating an auxiliary image for an image colorization process, the method comprising:
displaying an original image to a user; receiving a first user input determining a first region of the original image for which a target color is unspecified; receiving a second user input determining a target color for a second region of the original image; and generating an auxiliary image from the original image and the first and second user inputs.

13 . A non-transitory computer readable storage medium having stored therein instructions enabling generating an auxiliary image for an image colorization process, which when executed by a computer, cause the computer to:
display an original image to a user; receive a first user input determining a first region of the original image for which a target color is unspecified; receive a second user input determining a target color for a second region of the original image; and generate an auxiliary image from the original image and the first and second user inputs.

14 . An apparatus for generating an auxiliary image for an image colorization process, the apparatus comprising:
a display configured to display an original image to a user; a user interface configured to receive a first user input determining a first region of the original image for which a target color is unspecified and to receive a second user input determining a target color for a second region of the original image; and an image generator configured to generate an auxiliary image from the original image and the first and second user inputs.

15 . An apparatus for generating an auxiliary image for an image colorization process, the apparatus comprising a processing device and a memory device having stored therein instructions, which, when executed by the processing device, cause the apparatus to:
display an original image to a user; receive a first user input determining a first region of the original image for which a target color is unspecified; receive a second user input determining a target color for a second region of the original image; and generate an auxiliary image from the original image and the first and second user inputs.

16 . The non-transitory computer readable storage medium according to claim 9 , wherein the auxiliary image comprises information about a first type of pixels, for which a target color is specified, and a second type of pixels, for which no target color is specified.

17 . The non-transitory computer readable storage medium according to claim 16 , wherein for the first type of pixels, the target color of a pixel is one of the color of the pixel in the original image and a new color.

18 . The non-transitory computer readable storage medium according to claim 16 , wherein the Guided Image Filter is a Confidence-Aware Guided Image Filter.

19 . The non-transitory computer readable storage medium according to claim 18 , wherein the instructions cause the computer to assign a small confidence of ‘0’ or a value in the vicinity of ‘0’ to a pixel without a specified target color and to assign a larger confidence of ‘1’ or a value in the vicinity of ‘1’ to a pixel with a specified target color.

20 . The non-transitory computer readable storage medium according to claim 19 , wherein the instructions cause the computer to assign the larger confidence to a pixel having the color of the pixel in the original image as a specified target color if and only if a distance of the pixel to another pixel without a specified target color is smaller than a distance threshold, and to otherwise assign the smaller confidence to the pixel.

21 . The non-transi

Assignees

• Thomson Licensing

Inventors

• Hoerentrup Jobst
• Schlosser Markus

Classifications

• G06T11/10 Primary

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• G06T11/001

  Physics · mapped topic

• G06T5/20

  using local operators · CPC title

• G06T2207/20024

  Filtering details · CPC title

• G06T5/001

  Physics · mapped topic