Video color mapping using still image

Patent metadata

Field	Value
Publication number	US-11223809-B2
Application number	US-201916570613-A
Country	US
Kind code	B2
Filing date	Sep 13, 2019
Priority date	Sep 13, 2019
Publication date	Jan 11, 2022
Grant date	Jan 11, 2022

Abstract

Techniques described herein include capturing a plurality of frames associated with a video file, capturing an image corresponding to one of the plurality of frames, generating a color map between the captured image and a corresponding frame of the plurality of frames, where the color map is based on a color and a tone of the captured image and a color and tone of the corresponding frame, tone correcting one or more selected frames associated with the video file based on the color map, and color correcting the one or more selected frames based on the color map.

First claim

What is claimed is:

1. A method comprising:
capturing a plurality of frames associated with a video file;
identifying an image corresponding to one of the plurality of frames;
generating a color map based on a color and a tone of the identified image;
matching a portion of the color map to a portion of the corresponding frame of the plurality of frames;
tone correcting a selected frame from the plurality of frames associated with the video file based on a correctable quality parameter of the matched portion of the color map; and
color correcting the selected frame based on a correctable quality parameter of the matched portion of the color map.

2. The method of claim 1 , further comprising performing a post-capture process on the identified image to improve a visual quality of the image.

3. The method of claim 1 , wherein the color map includes at least one correctable quality parameter indicating a difference between the identified image and the corresponding frame.

4. The method of claim 1 , wherein
the matching of the portion of the color map to the portion of the corresponding frame of the plurality of frames includes matching at least one of a pixel, a block of pixels, a range of pixels, or a partition of the selected frame to at least one of a pixel, a block of pixels, a range of pixels, or a partition of the color map, the method further comprising:
using a value of the color map to perform the tone correcting of the selected frame, and
using the value of the color map to perform the color correcting of the selected frame.

5. The method of claim 1 , wherein:
the color map is partitioned into an M×N grid having M columns and N rows,
each partition includes data indicating a correction parameter, and
the correction parameter includes a value for gamma correction.

6. The method of claim 1 , wherein the color correcting of the selected frame includes performing a smoothing process to minimize color discontinuities in the selected frame.

7. The method of claim 1 , wherein the image and the frame to which the image corresponds are captured in a same instance.

8. The method of claim 1 , further comprising:
partitioning the identified image; and
partitioning the corresponding frame into a partitioned frame,
wherein generating the color map further includes:
determining a color variance between each partition of the image and a corresponding partition of the partitioned frame,
determining a tone variance between each partition of the image and a corresponding partition of the partitioned frame, and
wherein at least one of the determined color variance and the determined tone variance is based on a luminance difference or a chrominance difference.

9. A method comprising:
capturing a plurality of frames associated with a video file;
identifying an image corresponding to one of the plurality of frames;
generating a color map based on a color and a tone of the image, wherein the color map is partitioned with a partition matching a portion of a frame of the plurality of frames and including a correctable quality parameter;
generating a data structure including data representing the color map and data representing the partitioned color map; and
storing the data structure as metadata in a header associated with the video.

10. The method of claim 9 , further comprising performing a post-capture process on the identified image to improve a visual quality of the image.

11. The method of claim 9 , wherein the color map includes at least one correctable quality parameter difference between the image and the frame to which the image corresponds.

12. The method of claim 9 , wherein
generating the data structure includes partitioning the color map into an M×N grid having M columns and
N rows,
each partition includes data indicating a correction parameter,
the correction parameter includes a value for gamma correction.

13. The method of claim 9 , further comprising:
partitioning the identified image; and
partitioning the frame to which the image corresponds into a partitioned frame,
wherein generating the color map further includes:
determining a color variance between each partition of the image and a corresponding partition of the partitioned frame,
determining a tone variance between each partition of the image and a corresponding partition of the partitioned frame,
wherein at least one of the color variance and the tone variance is based on a luminance difference or a chrominance difference.

14. A method comprising:
receiving a video file including a plurality of frames and a header including a data structure including data representing a color map;
selecting a first frame of the plurality of frames, the first frame being associated with the data representing the color map;
selecting a second frame of the plurality of frames, the second frame being a target frame for color correction;
matching a portion of the color map to a portion of the second frame;
tone correcting the second frame based on a correctable quality parameter of the matched portion of the color map; and
color correcting the second frame based on a correctable quality parameter of the matched portion of the color map.

15. The method of claim 14 , wherein the matching of the portion of the color map to the portion of the second frame includes matching at least one of a pixel, a block of pixels, a range of pixels, or a partition of the second frame to at least one of a pixel, a block of pixels, a range of pixels, or a partition of the color map, the method further comprising:
using a value of the color map to perform the tone correcting of the second frame, and
using the value of the color map to perform the color correcting of the second frame.

16. The method of claim 14 , wherein
the color map is partitioned into an M×N grid having M columns and N rows,
each partition of the partitioned color map includes data indicating a correction parameter, and
the correction parameter includes a value for gamma correction.

17. The method of claim 14 , wherein color correcting the second frame includes performing a smoothing process to minimize color discontinuities.

18. The method of claim 14 , wherein color correcting the second frame includes performing a blending process to minimize color discontinuities.

19. The method of claim 14 , wherein color correcting the second frame includes using a trained convolutional neural network to minimize color variance between a partition of the second frame and a partition of the color map.

20. The method of claim 14 , wherein color correcting the second frame includes a performing a chrominance correction process using a mean value of a color space variable and a variance value of the color space variable.

Assignees

• Google Llc

Inventors

• Wei Li
• Hong Wei

Classifications

• G06T7/97 Primary

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• H04N23/81 Primary

  for suppressing or minimising disturbance in the image signal generation · CPC title

• H04N23/83

  specially adapted for colour signals · CPC title

• H04N21/44

  Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title