Method, system, and device for color measurement of a surface

Patent metadata

Field	Value
Publication number	US-11810329-B2
Application number	US-202016953029-A
Country	US
Kind code	B2
Filing date	Nov 19, 2020
Priority date	Nov 19, 2020
Publication date	Nov 7, 2023
Grant date	Nov 7, 2023

Abstract

Methods and systems for determining a surface color of a target surface under an environment with an environmental light source. A plurality of images of the target surface are captured as the target surface is illuminated with a variable intensity, constant color light source and a constant intensity, constant color environmental light source, wherein the intensity of the light source on the target surface is varied by a known amount between the capturing of the images. A color feature tensor, independent of the environmental light source, is extracted from the image data, and used to infer a surface color of the target surface.

First claim

The invention claimed is:

1. A method of measuring a color of a target surface using a color measuring device, comprising:
capturing a plurality of digital images of the target surface using a camera of the color measuring device as the target surface is illuminated with constant color light emitted by a controlled light source of the color measuring device that is in a fixed position relative to the camera and a constant intensity, constant color environmental light, wherein the intensity of the light illuminating the target surface is varied by a known amount between the capturing of the images;
generating, image data included in the plurality of images, a plurality of linear equations based on the image data from each image in the plurality of images, the image data comprising a sum of a product of the color of the controlled light source and a surface color of the target surface and a product of the environmental light source and the surface color of the target surface;
determining a color feature tensor that corresponds to the product of the color of the light emitted by the controlled light source and the surface color of the target surface from the plurality of linear equations using linear regression; and
inferring the surface color of the target surface based on the color feature tensor.

2. The method of claim 1 , further comprising varying the intensity of the light illuminating the target surface by changing a distance between the controlled light source and the target surface.

3. The method of claim 1 , wherein inferring the surface color of the target surface based on the color feature tensor comprises inputting the color feature tensor into a trained model for processing the color feature tensor and inferring the surface color of the target surface based the processed color feature tensor.

4. The method of claim 3 , wherein the trained model is a trained linear model, and the linear model is trained by:
initializing weights of a weight matrix;
receiving a batch of training samples of a training dataset, each training sample of the training dataset including an image of a surface and a ground truth surface color tensor representative of a true color of the surface;
for each respective sample of the batch of training samples:
processing the image of respective training sample to generate a color feature tensor that corresponds to a product of a training color of the controlled light source and the ground truth surface color;
computing a surface color tensor as a product of the color feature tensor and the weight matrix;
determining an error value as a difference between the surface color tensor and the ground truth surface color tensor;
updating the weights of the weight matrix based on the error value; and
receiving further batches of training samples until the weights of the weight matrix are optimized.

5. The method of claim 3 , wherein the trained model is approximated by a neural network, and the neural network is trained by:
initializing weights of the neural network;
receiving a batch of training samples of a training dataset, each training sample of the training dataset including an image of a surface and a ground truth surface color of the surface;
for each respective training sample:
processing the image of training sample to generate a color feature tensor that corresponds to a product of light emitted by the controlled light source and the ground truth surface color of the surface;
forward propagating the color feature tensor generated for the respective training sample through the neural network to infer a surface color for the training sample;
computing an error value for as a difference between the inferred surface color and the ground truth surface color; and
performing backpropagation to update the weights of the neural network based on the error value.

6. The method of claim 1 , wherein capturing comprises:
detecting, using a time-of-flight sensor, a distance between the color measuring device and the target surface at a first time;
detecting, using the time-of-flight sensor, a distance between the color measuring device and the target surface at a second time; and
when the distance between the color measuring device and the target surface has changed, controlling the controlled light source to emit light of the constant color and a constant intensity to illuminate the target surface and capturing a digital frame of the target surface.

7. The method of claim 1 , wherein the intensity of the light illuminating the target surface is varied by changing an amount of electrical power supplied to the controlled light source at a constant distance from the target surface.

8. The method of claim 1 , wherein the inferring further comprises converting the inferred surface color of the surface in a first color space to a surface color in a second color space.

9. The method of claim 3 , wherein the trained model is trained by:
dividing a color space into a plurality of color subspaces;
initializing weights of a subspace separation model and weights of a color subspace model for each of the plurality of color subspaces;
receiving a training sample comprising an image of a surface and a ground truth surface color of the surface;
processing the image of the training sample to generate a color feature tensor that corresponds to a product of light emitted by the controlled light source and the ground truth surface color;
computing a subspace color tensor by each of the color subspace models using the received color feature tensor;
generating a subspace weight tensor by the subspace separation model;
inferring an surface color by applying the subspace weight tensor to the subspace color tensors;
determining an error value as a difference between the inferred surface color and the ground truth surface color; and
performing backpropagation to update the weights of each of the plurality of color subspace models and weights of the subspace separation model.

10. The method claim 9 , wherein the color space is divided in accordance with a manual definition.

11. The method of claim 9 , wherein the plurality of color subspace models and subspace separation model are trained concurrently.

12. A color measurement device for measuring a target surface color of a target surface illuminated with a constant intensity, constant color environmental light, the device comprising:
an image acquisition device configured to capture a plurality of images of the target surface;
a light source configured to illuminate the target surface with a constant color light wherein the intensity of a light emitted from the light source is varied by a known amount between the capturing of successive images;
a color measurement system configured to:
generate, from image data from the plurality of images, a plurality of linear equations based on the image data from each image in the plurality of images, the image data comprising a sum of a product of the color of the controlled light source and a surface color of the target surface and a product of the environmental light source and the surface color of the target surface;
determine a color feature tensor that corresponds to the product of the color of the light emitted by the controlled light source and the surface color of the target surface from the plurality of linear equations using linear regression; and
infer the surface color of the target surface based on the color feature tensor.

13. The color measurement device of claim 12 , wherein the intensity of the light illuminating the target surface is varied by changing a distance between the light source and the target surface.

Assignees

• Yu Yuanhao
• Li Shuhao
• Lu Juwei
• Tang Jin
• Huawei Tech Co Ltd

Inventors

• Yu Yuanhao
• Li Shuhao
• Lu Juwei
• Tang Jin

Classifications

• G06T7/90 Primary

  Determination of colour characteristics · CPC title

• G06N3/084

  Backpropagation, e.g. using gradient descent · CPC title

• G06T7/521

  from laser ranging, e.g. using interferometry; from the projection of structured light · CPC title

• G06T7/557

  from light fields, e.g. from plenoptic cameras · CPC title