Image data alignment for mobile system sensors

Patent metadata

Field	Value
Publication number	US-12530743-B2
Application number	US-202318500278-A
Country	US
Kind code	B2
Filing date	Nov 2, 2023
Priority date	Nov 2, 2023
Publication date	Jan 20, 2026
Grant date	Jan 20, 2026

Abstract

A computer that includes a processor and a memory, the memory including instructions executable by the processor to determine a first prediction with a machine learning system based on receiving a first image from a first camera and determine a second prediction with a machine learning system based on receiving a second image from a second camera. When the first prediction does not equal the second prediction within a user determined tolerance, determine color consistency based on comparing pixel values from the first image with a threshold determined based on previously determined pixel values, determine color correction parameters by determining pixel statistics based on pixel values from the first image to include in an image signal processing system and apply the color correction parameters to a third image from the first camera by receiving the third image at the image signal processing system.

First claim

The invention claimed is:

1 . A system, comprising:
a computer that includes a processor and a memory, the memory including instructions executable by the processor to:
determine a first prediction with a machine learning system based on receiving a first image from a first camera;
determine a second prediction with the machine learning system based on receiving a second image from a second camera;
when the first prediction does not equal the second prediction within a user determined tolerance:
determine color consistency based on comparing pixel values from the first image with a threshold determined based on previously determined pixel values;
determine color consistency parameters by determining pixel statistics based on pixel values from the first image and the second image to include in an image signal processing system; and
apply the color consistency parameters to the second image from the second camera with the image signal processing system.

2 . The system of claim 1 , the instructions including further instructions to determine the threshold by varying pixel values in a training dataset image input to a machine learning system to determine when a prediction output changes based on the pixel values.

3 . The system of claim 1 , wherein the pixel statistics include a pixel mean and a pixel standard deviation.

4 . The system of claim 1 , wherein the color consistency parameters include one or more of lens shading, white balance, defect pixel, denoise, color interpolation, edge enhancement, color correction matrix, brightness/contrast, and gamma.

5 . The system of claim 1 , wherein the color consistency is based on determining one or more of camera color consistency, spatial color consistency, and temporal color consistency on the images.

6 . The system of claim 5 , wherein camera color consistency is determined by comparing images acquired by different cameras viewing the same scene with the same illumination.

7 . The system of claim 5 , wherein spatial color consistency is determined by comparing overlapping images acquired by different cameras viewing portions of the same scene with differing illumination.

8 . The system of claim 5 , wherein temporal color consistency is determined by comparing images acquired by the camera viewing the same scene at different times.

9 . The system of claim 1 , wherein the first prediction and the second prediction include one or more of object identity and object location.

10 . The system of claim 9 , wherein the object identity includes one or more of a roadway and a vehicle.

11 . The system of claim 1 , wherein the machine learning system includes a convolutional neural network that includes convolutional layers and fully connected layers.

12 . The system of claim 1 , the instructions including further instructions to convert a red, green, blue (RGB) color space image to a luma, red projection, blue projection (YUV) image and outputting the image.

13 . The system of claim 1 , wherein the machine learning system is included in a mobile machine.

14 . The system of claim 13 , wherein the mobile machine is operated based on the one or more predictions.

15 . The system of claim 14 , wherein the mobile machine is a vehicle and the vehicle is operated by controlling one or more of vehicle propulsion and vehicle steering.

16 . A method, comprising:
determining a first prediction with a machine learning system based on receiving a first image from a first camera; determining a second prediction with the machine learning system based on receiving a second image from a second camera; when the first prediction does not equal the second prediction within a user determined tolerance:
determining color consistency based on comparing pixel values from the first image with a threshold determined based on previously determined pixel values;
determining color consistency parameters by determining pixel statistics based on pixel values from the first image to include in an image signal processing system; and
applying the color consistency parameters to a third image from the first camera by receiving the third image at the image signal processing system.

17 . The method of claim 16 , further comprising determining the threshold by varying pixel values in a training dataset image input to a machine learning system to determine when a prediction output changes based on the pixel values.

18 . The method of claim 16 , wherein the pixel statistics include a pixel mean and a pixel standard deviation.

19 . The method of claim 16 , wherein the color consistency parameters include one or more of lens shading, white balance, defect pixel, denoise, color interpolation, edge enhancement, color correction matrix, brightness/contrast, and gamma.

20 . The method of claim 16 , wherein the color consistency is based on determining one or more of camera color consistency, spatial color consistency, and temporal color consistency on the images.

Assignees

• Ford Global Tech Llc

Inventors

• Herman david michael
• Sanders Larry Dean
• Vaidya Akshay

Classifications

• B60W2420/403

  Image sensing, e.g. optical camera · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• G06T2207/30256

  Lane; Road marking · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T2207/10152

  Varying illumination · CPC title