Protocol for high integrity personal electronic device applications

Patent metadata

Field	Value
Publication number	US-10901674-B2
Application number	US-201815959510-A
Country	US
Kind code	B2
Filing date	Apr 23, 2018
Priority date	Dec 7, 2017
Publication date	Jan 26, 2021
Grant date	Jan 26, 2021

Abstract

A method in a high integrity device for verifying the integrity, continuity, and availability (ICA) of information displayed on an uncertified display is provided. The method comprises creating, on the high integrity device, application data for display on an uncertified display device; transmitting the application data to the uncertified display device; receiving, at the high integrity device from an optical sensing device, a captured image of the display; determining, at the high integrity device, an expected display image to be displayed on the uncertified display device; comparing, at the high integrity device, the captured image of the display to the expected display image; identifying, at the high integrity device, a loss of ICA if one or more frames in the captured image fails the comparison with the expected display image; and causing the annunciation of a message indicating the loss of ICA when a loss of ICA is identified.

First claim

What is claimed is:

1. A system for verifying the integrity, continuity, and availability (ICA) of information displayed on an uncertified display, the system comprising:
a server that is certified to generate mission critical data, the server comprising one or more processors configured by programming instructions on computer readable media and an application module configured to generate mission critical data for display; and
an adapter device configured to house an uncertified display device that provides the uncertified display, the adapter device comprising an optical sensing device configured to capture an image displayed on the uncertified display and an annunciation screen configured to annunciate a message indicating a loss of ICA when a loss of ICA is detected;
wherein the system is configured to:
create, on the server, application data for display on the uncertified display device;
transmit the application data to the uncertified display device for display;
capture, using the optical sensing device, an image of the display screen of the uncertified display device;
determine, at the server, an expected display image to be displayed on the uncertified display device;
compare, at the server, the captured image of the display to the expected display image; and
identify a loss of ICA when one or more frames in the captured image fails the comparison with the expected display image.

2. The system of claim 1 , wherein the system is further configured to transmit an error indicator, from the server to the adapter device, when a loss of ICA is identified.

3. The system of claim 2 , wherein the system is further configured to display on the annunciation screen a message indicating a loss of ICA when a loss of ICA is identified.

4. The system of claim 3 , wherein the annunciation screen is configured to overlay the display screen of the uncertified display device.

5. The system of claim 1 , wherein to compare the captured image of the display to the expected display image, the system is configured to calculate an image characteristic for both the expected display image and the captured image.

6. The system of claim 5 , wherein to compare the captured image of the display to the expected display image, the system is configured to calculate a similarity measure between the expected display image and the captured image.

7. The system of claim 5 , wherein to compare the captured image of the display to the expected display image, the system is configured to calculate histograms for each of the captured image and the expected image.

8. The system of claim 5 , wherein to compare the captured image of the display to the expected display image, the system is configured to implement an artificial neural network (ANN) to calculate image characteristics.

9. The system of claim 1 , wherein the server is further configured to:
embed special display frames with frames of application data before the application data is transmitted to the uncertified display device for display;
compare the captured image of the display to the expected display image by examining the captured image of the display to identify the embedded special display frames; and
identify a loss of ICA when one or more of the embedded special display frames are not found or are incorrect in the captured image.

10. The system of claim 1 , wherein to compare the captured image of the display to the expected display image, the server is configured to compare critical fields and not all fields of the captured image of the display to the expected display image.

11. The system of claim 1 , wherein to compare the captured image of the display to the expected display image, the server is configured to compare the captured image of the display to the expected display image using pixel matching.

12. A processor-implemented method in a mission critical device for verifying the integrity, continuity, and availability (ICA) of information displayed on an uncertified display, the method comprising:
creating, on the mission critical device by a processor, application data for display on an uncertified display device;
transmitting the application data to the uncertified display device for display;
receiving, at the mission critical device, a captured image of the display from an optical sensing device trained on the display screen of the uncertified display device, wherein the optical sensing device is on an adapter device that houses the uncertified display device;
determining, at the mission critical device, an expected display image to be displayed on the uncertified display device;
comparing, at the mission critical device, the captured image of the display to the expected display image;
identifying, at the mission critical device, a loss of ICA if one or more frames in the captured image fails the comparison with the expected display image; and
causing the annunciation of a message indicating the loss of ICA when a loss of ICA is identified.

13. The method of claim 12 , wherein comparing the captured image of the display to the expected display image comprises calculating an image characteristic for both the expected display image and the captured image.

14. The method of claim 13 , wherein comparing the captured image of the display to the expected display image comprises calculating a similarity measure between the expected display image and the captured image.

15. The method of claim 13 , wherein comparing the captured image of the display to the expected display image comprises calculating histograms for each of the captured image and the expected image.

16. The method of claim 13 , wherein comparing the captured image of the display to the expected display image comprises implementing an artificial neural network (ANN) to calculate image characteristics.

17. The method of claim 12 , wherein comparing the captured image of the display to the expected display image comprises comparing one or more critical fields but less than the complete captured display image of the display to the expected display image for those one or more critical fields.

18. A method of verifying the integrity, continuity, and availability (ICA) of information displayed on an uncertified display, the method comprising:
creating, on a mission critical device, application data for display on an uncertified display device;
periodically embedding special display frames in the application data;
transmitting the application data with the embedded special display frames to the uncertified display device for display;
receiving a captured image of the display from an optical sensing device trained on the display screen of the uncertified display device, wherein the optical sensing device is on an adapter device that houses the uncertified display device;
examining the captured image of the display to identify the embedded special display frames; and
identifying a loss of ICA if one or more of the embedded special display frames are not found or are incorrect in the captured image.

19. The method of claim 18 , further comprising varying features of the special display frames from one special display frame to another.

20. The method of claim 18 , further comprising requiring multiple embedded special display frame failures before identifying a loss of ICA.

Assignees

• Honeywell Int Inc

Inventors

• True Willard R
• Berckefeldt Richard
• Sarma Kalluri R

Classifications

• G09G2380/12

  Avionics applications · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06F3/1415 Primary

  with means for detecting differences between the image stored in the host and the images displayed on the displays · CPC title

• G06F3/147

  using display panels · CPC title

• G06F21/50

  Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems · CPC title