Safety monitor for invalid image transform

Patent metadata

Field	Value
Publication number	US-2022100634-A1
Application number	US-202117548393-A
Country	US
Kind code	A1
Filing date	Dec 10, 2021
Priority date	May 31, 2019
Publication date	Mar 31, 2022
Grant date	—

Abstract

Systems, apparatuses, and methods for implementing a safety monitor framework for a safety-critical computer vision (CV) application are disclosed. A system includes a safety-critical CV application, a safety monitor, and a CV accelerator engine. The safety monitor receives an input image, test data, and a CV graph from the safety-critical CV application. The safety monitor generates a modified image by adding additional objects outside of the input image. The safety monitor provides the modified image and CV graph to the CV accelerator which processes the modified image and provides outputs to the safety monitor. The safety monitor determines the likelihood of erroneous processing of the original input image by comparing the outputs for the additional objects with a known good result. The safety monitor complements the overall fault coverage of the CV accelerator engine and covers faults only observable at the level of the CV graph.

First claim

What is claimed is:

1 . A system comprising:
a computer vision accelerator engine; and a safety monitor configured to:
receive an input image and test data from a safety-critical application;
generate a modified image by combining one or more given objects with the input image;
convey the modified image to the computer vision accelerator engine; and
generate a confidence indicator based on an analysis of results generated by the computer vision acceleration engine processing the modified image, wherein the confidence indicator represents a probability that the results are accurate.

2 . The system as recited in claim 1 , wherein the system is configured to perform one or more corrective actions in response to the confidence indicator not meeting a threshold.

3 . The system as recited in claim 2 , wherein the one or more corrective actions comprise terminating the safety-critical application.

4 . The system as recited in claim 1 , wherein:
the test data indicates how the one or more given objects should be processed by the computer vision accelerator; and the safety monitor is configured to analyze the results to determine if the one or more given objects were processed correctly.

5 . The system as recited in claim 1 , wherein the safety monitor is further configured to:
analyze a previous input image to detect at least one known good object; add a redundant version of the at least one known good object to extra space outside of original boundaries of the input image; and create the modified image from the input image and the extra space.

6 . The system as recited in claim 1 , wherein the safety monitor is further configured to:
receive a first set of output data from the computer vision accelerator engine; convert the first set of output data to a second set of output data; and convey the second set of output data to the safety-critical application executing on the one or more processors.

7 . The system as recited in claim 1 , wherein the confidence indicator specifies whether the computer vision accelerator engine passed or failed verification of the results, and wherein the one or more given objects are selected based on the test data received from the safety-critical application.

8 . A method comprising:
receiving, by a safety monitor, an input image and test data from a safety-critical application; generating, by the safety monitor, a modified image by combining one or more given objects with the input image; conveying the modified image to a computer vision accelerator engine; generating, by the safety monitor, a confidence indicator based on an analysis of results generated by the computer vision acceleration engine processing the modified image, wherein the confidence indicator represents a probability that the results are accurate; and performing, by the safety-critical application, one or more corrective actions in response to the confidence indicator being below a threshold.

9 . The method as recited in claim 8 , wherein the confidence indicator being below a threshold indicates that the computer vision accelerator engine is malfunctioning.

10 . The method as recited in claim 8 , wherein the one or more corrective actions comprise terminating the safety-critical application.

11 . The method as recited in claim 8 , further comprising analyzing, by the safety monitor, the results to determine if the one or more given objects were processed correctly, wherein the test data indicates how the one or more given objects should be processed by the computer vision accelerator.

12 . The method as recited in claim 8 , further comprising:
analyzing, by the safety monitor, a previous input image to detect at least one known good object; and adding, by the safety monitor, a redundant version of the at least one known good object to extra space outside of original boundaries of the input image; and creating, by the safety monitor, the modified image from the input image and the extra space.

13 . The method as recited in claim 8 , further comprising:
receiving, by the safety monitor, a first set of output data from the computer vision accelerator engine; and converting, by the safety monitor, the first set of output data to a second set of output data; and conveying, by the safety monitor, the second set of output data to the safety-critical application.

14 . The method as recited in claim 8 , wherein the confidence indicator specifies whether the computer vision accelerator engine passed or failed verification of the results, and wherein the one or more given objects are selected based on the test data received from the safety-critical application.

15 . An apparatus comprising:
a memory storing program instructions; and at least one processor coupled to the memory, wherein the program instructions are executable by the at least one processor to:
receive an input image and test data from a safety-critical application executing on the one or more processing units;
generate a modified image by combining one or more given objects with the input image;
convey the modified image to a computer vision accelerator engine;
generate a confidence indicator based on an analysis of results generated by the computer vision acceleration engine processing the modified image, wherein the confidence indicator represents a probability that the results are accurate; and
perform one or more corrective actions in response to the confidence indicator being below a threshold.

16 . The apparatus as recited in claim 15 , wherein the confidence indicator being below a threshold indicates that the computer vision accelerator engine is malfunctioning.

17 . The apparatus as recited in claim 15 , wherein the one or more corrective actions comprise terminating a safety-critical application.

18 . The apparatus as recited in claim 15 , wherein the program instructions are further executable by the at least one processor to analyze the results to determine if the one or more given objects were processed correctly, wherein the test data indicates how the one or more given objects should be processed by the computer vision accelerator.

19 . The apparatus as recited in claim 15 , wherein the program instructions are further executable by the at least one processor to:
analyze a previous input image to detect at least one known good object; add a redundant version of the at least one known good object to extra space outside of original boundaries of the input image; and create the modified image from the input image and the extra space.

20 . The apparatus as recited in claim 15 , wherein the program instructions are further executable by the at least one processor to:
receive a first set of output data from the computer vision accelerator engine; and convert the first set of output data to a second set of output data.

Assignees

• Ati Technologies Ulc

Inventors

• Kwong Tung Chuen
• Chan Benjamin Koon Pan
• Ip Clarence
• Manjunatha Meghana

Classifications

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• G06T7/0002

  Inspection of images, e.g. flaw detection · CPC title

• G06T3/4038

  Image mosaicing, e.g. composing plane images from plane sub-images · CPC title

• G06T1/0014

  Image feed-back for automatic industrial control, e.g. robot with camera (robots B25J19/023) · CPC title

• G06F11/3692

  for test results analysis · CPC title