Color balancing based on reference points

Patent metadata

Field	Value
Publication number	US-9491428-B2
Application number	US-201314086660-A
Country	US
Kind code	B2
Filing date	Nov 21, 2013
Priority date	Apr 29, 2013
Publication date	Nov 8, 2016
Grant date	Nov 8, 2016

Abstract

Embodiments provide techniques for adjusting coloration of an image. A visual scene is captured using one or more camera devices, where the visual scene includes a reference object. Embodiments retrieve coloration information corresponding to the reference object. The coloration information generally describes coloration of the reference object under predefined conditions, such as neutral lighting conditions, desired lighting conditions, and so on. Additionally, a coloration difference is calculated between a depiction of the reference object within the captured visual scene and the retrieved coloration information. The coloration of the visual scene is adjusted based on the determined coloration difference.

First claim

What is claimed is:

1. A method of adjusting coloration, comprising:
identifying at least one reference point within a visual scene;
retrieving, from a non-transitory memory device, profile information corresponding to the at least one reference point, wherein the profile information describes coloration of the at least one reference point under predefined conditions;
determining a coloration difference between a depiction of the at least one reference point within the visual scene and the retrieved profile information; and
adjusting, by operation of one or more computer processors, the coloration of the visual scene, based on the determined coloration difference.

2. The method of claim 1 , wherein the at least one reference point corresponds to a reference object within the visual scene, and wherein the profile information describes coloration of the reference object under predefined conditions.

3. The method of claim 2 , wherein identifying the at least one reference point within the visual scene further comprises:
identifying the reference object within the visual scene;
determining a plurality of potential reference points within the visual scene corresponding to the identified reference object; and
determining the at least one reference point within the visual scene by selecting the at least one reference point from the plurality of potential reference points.

4. The method of claim 3 , wherein identifying the at least one reference point within the visual scene further comprises:
excluding one or more outlier points from the plurality of potential reference points from the selection of the at least one reference point.

5. The method of claim 3 , wherein determining the at least one reference point within the visual scene by selecting one of the plurality of potential reference points further comprises:
receiving a first user selection specifying the at least one reference point; and
receiving a second user selection of a second at least one reference point within the retrieved profile information that corresponds to the selected at least one reference point, and
wherein determining the coloration difference further comprises comparing a coloration of the at least one reference point within the visual scene with a second coloration of the second at least one reference point within the retrieved profile information.

6. The method of claim 5 , wherein the first user selection specifies a plurality of reference points within the visual scene, wherein the second user selection specifies a second plurality of reference points within the retrieved profile information, and wherein each of the plurality of reference points maps to a respective one of the second plurality of reference points.

7. The method of claim 1 , wherein the captured visual scene comprises a single frame of a first video stream.

8. The method of claim 7 , further comprising:
receiving a second video stream to be composited with the first video stream;
determining a second coloration difference between a depiction of a second at least one reference point within a frame of the second video stream and the retrieved profile information and
adjusting coloration of at least a portion of the frame of the second video stream, based on the determined second coloration difference.

9. The method of claim 1 , wherein adjusting the coloration of the visual scene further comprises:
rendering one or more frames depicting a virtual object inserted into the visual scene, wherein a coloration of the virtual object is adjusted based on the determined coloration difference.

10. The method of claim 9 , wherein identifying the at least one reference point within the visual scene comprises identifying a first plurality of reference points within the visual scene, and the method further comprising:
identifying a second plurality of reference points within the retrieved profile information, wherein each of the second plurality of reference points corresponds to a respective one of the first plurality of reference points, and
wherein determining the coloration difference is based on both the first plurality of reference points and the second plurality of reference points.

11. The method of claim 1 , further comprising:
wherein identifying the at least one reference point within the visual scene comprises identifying a first plurality of reference points within the visual scene, and further comprising:
for each of the first plurality of reference points, determining a corresponding reference point within the retrieved profile information, and
wherein determining the coloration difference further comprises:
determining a plurality of coloration differences by, for each of the first plurality of reference points, determining a respective coloration difference between the reference point from the first plurality of reference points and a corresponding reference point from the second plurality of reference points; and
determining an aggregate coloration difference for the visual scene, based on the plurality of coloration differences.

12. A system, comprising:
one or more computer processors; and
a non-transitory memory containing a program that, when executed by the one or more computer processors, performs an operation for adjusting coloration, comprising:
identifying at least one reference point within a visual scene;
retrieving profile information corresponding to the at least one reference point, wherein the profile information describes coloration of the at least one reference point under predefined conditions;
determining a coloration difference between a depiction of the at least one reference point within the visual scene and the retrieved profile information; and
adjusting the coloration of the visual scene, based on the determined coloration difference.

13. The system of claim 12 , wherein the captured visual scene comprises a single frame of a first video stream, and the operation further comprising:
receiving a second video stream to be composited with the first video stream;
determining a second coloration difference between a depiction of a second at least one reference point within a frame of the second video stream and the retrieved profile information and
adjusting coloration of at least a portion of the frame of the second video stream, based on the determined second coloration difference.

14. The system of claim 12 , wherein identifying the at least one reference point within the visual scene comprises identifying a first plurality of reference points within the visual scene, and the operation further comprising:
identifying a second plurality of reference points within the retrieved profile information, wherein each of the second plurality of reference points corresponds to a respective one of the first plurality of reference points,
wherein determining the coloration difference is based on both the first plurality of reference points and the second plurality of reference points, and
wherein adjusting the coloration of the visual scene further comprises rendering one or more frames depicting a virtual object inserted into the visual scene, wherein a coloration of the virtual object is adjusted based on the determined coloration difference.

15. A non-transitory computer-readable medium containing computer program code that, when executed, performs an operation for adjusting coloration, the operation comprising:
identifying at least one reference point within a visual scene;
retrieving profile information corresponding to the at least one reference point, wherein the profile information describes coloration o

Assignees

• Disney Entpr Inc
• Eth Zurich (Eidgenoessische Technische Hochschule Zurich)

Inventors

• Sorkine Hornung Alexander
• Oskam Thomas
• Sumner Robert Walker

Classifications

• H04N1/6077 Primary

  Colour balance, e.g. colour cast correction · CPC title

• H04N23/88

  for colour balance, e.g. white-balance circuits or colour temperature control · CPC title

• H04N1/603

  controlled by characteristics of the picture signal generator or the picture reproducer · CPC title

• H04N9/73 Primary

  Colour balance circuits, e.g. white balance circuits or colour temperature control (camera processing pipelines for colour balance H04N23/88) · CPC title