Integrity monitoring systems and methods for image sensors

Patent metadata

Field	Value
Publication number	US-10907968-B1
Application number	US-201815882885-A
Country	US
Kind code	B1
Filing date	Jan 29, 2018
Priority date	Jan 29, 2018
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

An integrity monitoring system for a first image sensor includes an electronic processor configured to receive sensor data for the provision of image data associated with an environment of the avionic sensor. The electronic processor is configured to monitor the avionic sensor for integrity. The electronic processor is configured to perform at least one of: determining a presence of an optical feature associated with optics of the first image sensor, comparing overlap information derived from the sensor data and other sensor data, comparing characteristics of a digital output stream of the sensor data to expected characteristics, or comparing a first motion derived from the image data and a second motion derived from avionic position equipment.

First claim

What is claimed is:

1. An integrity monitoring system for a first image sensor, the integrity monitoring system comprising:
an electronic processor configured to receive sensor data for provision of image data associated with an environment of the first image sensor, wherein the electronic processor is configured to monitor the first image sensor for integrity, wherein the electronic processor is configured to determine a presence of an optical feature associated with optics of the first image sensor in the sensor data and compare a location of the optical feature with an expected location to determine the integrity of the first image sensor and the optical feature is a natural or artificial physical defect or marking in a lens or lens cover, wherein first motion data associated with the image data is determined after de-noising, wherein the de-noising comprises excluding data from blocks that do not match and averaging block motion across frames at different times.

2. The integrity monitoring system of claim 1 , wherein the optical feature is an artificial marking in the lens or lens cover.

3. The integrity monitoring system of claim 1 , further comprising a sensor inductively coupled to the first image sensor to detect a digital output stream, and wherein the electronic processor compares characteristics of the digital output stream of the sensor data to expected characteristics to determine the integrity of the first image sensor.

4. The integrity monitoring system of claim 1 , wherein the electronic processor is configured to perform at least two of:
a. determining the presence of the optical feature associated with the optics of the first image sensor in the sensor data and comparing the location or shape of the optical feature with the expected location to determine the integrity of the first image sensor;
b. comparing overlap information derived from the sensor data and from a second image sensor to determine the integrity of the first image sensor;
c. comparing characteristics of a digital output stream of the sensor data to expected characteristics to determine the integrity of the first image sensor; and
d. comparing a first motion derived from the image data and a second motion derived from an avionic position equipment to determine the integrity of the first image sensor.

5. The integrity monitoring system of claim 1 , wherein the electronic processor is configured to compensate for distortion associated with the optical feature.

6. The integrity monitoring system of claim 5 , wherein electronic processor is configured to use neighboring pixels to remove the optical feature from being depicted in an image associated with the image data.

7. The integrity monitoring system of claim 1 , wherein the optical feature further comprises a light source, the light source being periodically on and the electronic processor is configured to detect a frozen frame error is the light source is detected as on in an image associated with the image data when the light source is not on.

8. The integrity monitoring system of claim 1 , wherein the optical feature comprises multiple optical features disposed at different known locations in an image associated with the image data.

9. The integrity monitoring system of claim 1 , wherein a size and a shape of the optical feature is compared to a known size and shape to determine the integrity.

10. The integrity monitoring system of claim 1 , further comprising a light source, wherein the electronic processor is configured to control the light source to periodically illuminate the optical feature so that it can be viewed in night or foggy conditions.

11. The integrity monitoring system of claim 5 , wherein the optical feature is a scratch on the lens.

12. An integrity monitoring system for a first image sensor, the integrity monitoring system comprising:
an electronic processor configured to receive sensor data for provision of image data associated with an environment of the first image sensor, wherein the electronic processor is configured to monitor the first image sensor for integrity,
wherein the electronic processor is configured to perform:
a. determining a presence of an optical feature associated with optics of the first image sensor in the sensor data and comparing a location or shape of the optical feature with an expected location to determine the integrity of the first image sensor
b. comparing overlap information derived from the sensor data and from another image sensor to determine the integrity of the first image sensor;
c. comparing characteristics of a digital output stream of the sensor data to expected characteristics to determine the integrity of the first image sensor; and
d. comparing a first motion derived from the image data and a second motion derived from an avionic position equipment to determine the integrity of the first image sensor.

13. The integrity monitoring system of claim 12 , wherein the first image sensor is an avionic camera.

14. The integrity monitoring system of claim 12 , wherein the first image sensor is a visible light avionic camera.

15. A flight control center, comprising:
an image sensor configured to provide sensor data associated with an image;
an electronic processor configured to receive the sensor data, wherein the electronic processor is configured to monitor the image sensor for an error, wherein the electronic processor determines a presence of an optical feature associated with optics of the image sensor in the sensor data and compares a location of the optical feature with an expected location to determine a presence of the error and the optical feature is a physical defect provided in a transmissive optical element of a commercial off the shelf camera, wherein the optical feature further comprises a light source, the light source being periodically on and the electronic processor is configured to detect a frozen frame error as the error when the light source is detected as on in the image when the light source is not on.

16. The flight control center of claim 15 , wherein the defect is provided in a lens or cover of the commercial off the shelf camera.

17. The flight control center of claim 16 , wherein the electronic processor compares characteristics of a digital output stream associated with the sensor data to expected characteristics to determine the presence of the error and the characteristics comprise a horizontal sync signal, a vertical sync signal or a pixel clock.

18. The flight control center of claim 16 , wherein the electronic processor compares characteristics of a digital output stream associated with the sensor data to expected characteristics to determine the presence of the error and the characteristics are a known pixel pattern.

19. The flight control center of claim 15 , wherein first motion data associated with the image data is determined after de-noising, wherein the de-noising comprises excluding data from blocks that do not match and averaging block motion across frames at different times.

20. A flight control center, comprising:
an image sensor configured to provide sensor data associated with an image;
an electronic processor configured to receive the sensor data, wherein the electronic processor is configured to monitor the image sensor for an error, wherein the electronic processor determines a presence of an optical feature associated with optics of the image sensor in the sensor data and compares a location of the optical feature with an expected location to determine a presence of the er

Assignees

• Rockwell Collins Inc

Inventors

• Tiana Carlo L
• Brown Robert D
• Lahr Weston J
• Chiew Daniel
• Barnes Elisabeth
• Murthy Yashaswi M Narasimha
• Wilson Brandon E
• Wood Robert W

Classifications

• H04N25/68

  applied to defects · CPC title

• H04N23/6811

  based on the image signal · CPC title

• H04N23/6812

  based on additional sensors, e.g. acceleration sensors · CPC title

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/60

  Control of cameras or camera modules · CPC title