3D Color Mapping and Tuning in an Image Processing Pipeline

Patent metadata

Field	Value
Publication number	US-2017359488-A1
Application number	US-201615181143-A
Country	US
Kind code	A1
Filing date	Jun 13, 2016
Priority date	Jun 13, 2016
Publication date	Dec 14, 2017
Grant date	—

Abstract

The processing of RGB image data can be optimized by performing optimization operations on the image data when it is converted into the YCbCr color space. First, a raw RGB color space is converted into a YCbCr color space, and raw RGB image data is converted into YCbCr image data using the YCbCr color space. For each Y-layer of the YCbCr image data, a 2D LUT is generated. The YCbCr image data is converted into optimized CbCr image data using the 2D LUTs, and optimized YCbCr image data is generated by blending CbCr image data corresponding to multiple Y-layers. The optimized YCbCr image data is converted into sRGB image data, and a tone curve is applied to the sRGB image data to produce optimized sRGB image data.

First claim

What is claimed is:

1 . A camera system, comprising:
an image sensor configured to convert light incident upon the image sensor into RGB image data; an image signal processor (“ISP”) configured to:
access a set of linear 3D color points corresponding to an input color space;
generate a color-mapped linear 3D color look-up table (“LUT”) based on the set of linear 3D color points;
generate a gain curve based on the RGB image data; and
apply the gain curve to the color mapped linear 3D color look-up table to generate a tone-mapped color-mapped linear 3D color LUT; and
a memory configured to store the tone-mapped color-mapped linear 3D color LUT.

2 . The camera system of claim 1 , wherein generating a color-mapped linear 3D color LUT comprises:
applying a first 3D color LUT with a fixed encoding gamma curve to the set of linear 3D color points to generate a set of non-linear color-mapped color points.

3 . The camera system of claim 2 , wherein generating a color-mapped linear 3D color LUT further comprises:
applying a decoding gamma curve to the non-linear color-mapped color points to generate the color-mapped linear 3D color points.

4 . The camera system of claim 1 , wherein applying the gain curve to the color-mapped linear 3D color LUT comprises:
determining a luminance intensity of each color in the color-mapped linear 3D color LUT; and applying the luminance intensities to the color-mapped linear 3D color LUT.

5 . The camera system of claim 1 , wherein generating a gain curve based on the RGB image data comprises generating a gain curve based on a downscaled version of the RGB image data.

6 . The camera system of claim 1 , wherein generating a gain curve based on the RGB image data comprises:
applying a first 3D color LUT with a fixed encoding gamma curve to the RGB image data to produce nonlinear color-mapped color points; and applying a decoding gamma curve to the nonlinear color-mapped color points to produce color-mapped linear RGB image data.

7 . The camera system of claim 6 , wherein generating a gain curve based on the RGB image data further comprises:
applying a global tone mapping to the color-mapped linear RGB image data to produce the gain curve.

8 . The camera system of claim 1 , wherein the ISP is further configured to:
apply an encoding gamma curve to the tone-mapped color-mapped linear 3D color LUT to produce a tone-mapped color-mapped nonlinear 3D color LUT.

9 . The camera system of claim 8 , wherein the ISP is further configured to:
apply the tone-mapped color-mapped nonlinear 3D color LUT to the RGB image data to produce a tone-mapped and color-mapped image.

10 . The camera system of claim 9 , wherein the camera system further comprises an output configured to output the tone-mapped and color-mapped image.

11 . A method, comprising:
converting, by a camera, light incident upon an image sensor of the camera into raw RGB image data; accessing, by an image signal processor (“ISP”) of the camera, a set of linear 3D color points corresponding to an input color space; generating, by the ISP of the camera, a color-mapped linear 3D color look-up table (“LUT”) based on the set of linear 3D color points; generating, by the ISP of the camera, a gain curve based on the RGB image data; and applying, by the ISP of the camera, the gain curve to the color mapped linear 3D color look-up table to generate a tone-mapped color-mapped linear 3D color LUT; and storing, by a memory of the camera, the tone-mapped color-mapped linear 3D color LUT.

12 . The method of claim 11 , wherein generating a color-mapped linear 3D color LUT comprises:
applying a first 3D color LUT with a fixed encoding gamma curve to the set of linear 3D color points to generate a set of non-linear color-mapped color points.

13 . The method of claim 12 , wherein generating a color-mapped linear 3D color LUT further comprises:
applying a decoding gamma curve to the non-linear color-mapped color points to generate the color-mapped linear 3D color points.

14 . The method of claim 11 , wherein applying the gain curve to the color-mapped linear 3D color LUT comprises:
determining a luminance intensity of each color in the color-mapped linear 3D color LUT; and applying the luminance intensities to the color-mapped linear 3D color LUT.

15 . The method of claim 11 , wherein generating a gain curve based on the RGB image data comprises generating a gain curve based on a downscaled version of the RGB image data.

16 . The method of claim 11 , wherein generating a gain curve based on the RGB image data comprises:
applying a first 3D color LUT with a fixed encoding gamma curve to the RGB image data to produce nonlinear color-mapped color points; and applying a decoding gamma curve to the nonlinear color-mapped color points to produce color-mapped linear RGB image data.

17 . The method of claim 16 , wherein generating a gain curve based on the RGB image data further comprises:
applying a global tone mapping to the color-mapped linear RGB image data to produce the gain curve.

18 . The method of claim 11 , further comprising:
applying, by the ISP of the camera, an encoding gamma curve to the tone-mapped color-mapped linear 3D color LUT to produce a tone-mapped color-mapped nonlinear 3D color LUT.

19 . The method of claim 18 , further comprising:
applying, by the ISP of the camera, the tone-mapped color-mapped nonlinear 3D color LUT to the RGB image data to produce a tone-mapped and color-mapped image.

20 . The method of claim 19 , further comprising:
outputting, by an output of the camera, the tone-mapped and color-mapped image.

Assignees

• Gopro Inc

Inventors

• Chen Ying
• Chawla Sumit
• Edwards Elizabeth Torres
• Chandu Kartheek
• Ding Bo

Classifications

• H04N1/6019 Primary

  using look-up tables (H04N1/6025 takes precedence) · CPC title

• H04N1/6008

  with primary colour signals, e.g. RGB or CMY(K) · CPC title

• H04N1/6005

  with luminance or chrominance signals, e.g. LC1C2, HSL or YUV · CPC title

• G06T1/20

  Processor architectures; Processor configuration, e.g. pipelining · CPC title

• H04N23/86

  for controlling the colour saturation of colour signals, e.g. automatic chroma control circuits · CPC title