Method and apparatus with test result reliability verification

Patent metadata

Field	Value
Publication number	US-12517799-B2
Application number	US-202418828022-A
Country	US
Kind code	B2
Filing date	Sep 9, 2024
Priority date	Sep 8, 2023
Publication date	Jan 6, 2026
Grant date	Jan 6, 2026

Abstract

A method for verifying reliability of a test for products performed by test equipment includes: receiving result images generated from preprocessing of result data of the test, the result data of the test including labels for a plurality of scale levels and the received result images including first result images belonging to a first scale level of the plurality of scale levels and second result images belonging to a second scale level of the plurality of scale levels; making a first determination, from the first result images, whether the first scale level is normal or abnormal; making a second determination, from the second result images, whether the second scale level is normal or abnormal; and determining that no error occurred in the test in response to both the first scale level and the second scale level being determined to be normal; or determining that an error occurred in the test in response to at least one scale level being determined to be abnormal.

First claim

What is claimed is:

1 . A method for verifying reliability of a test for products performed by test equipment, the method comprising:
receiving result images generated from preprocessing of result data of the test, the result data of the test including labels for a plurality of scale levels and the received result images including first result images belonging to a first scale level of the plurality of scale levels and second result images belonging to a second scale level of the plurality of scale levels; making a first determination, from the first result images, whether the first scale level is normal or abnormal; making a second determination, from the second result images, whether the second scale level is normal or abnormal; and determining that no error occurred in the test in response to both the first scale level and the second scale level being determined to be normal; or determining that an error occurred in the test in response to at least one scale level being determined to be abnormal.

2 . The method of claim 1 , wherein each label includes an identifier of a scale level.

3 . The method of claim 2 , further comprising classifying the result images by their corresponding scale levels according to the labels.

4 . The method of claim 1 , wherein element areas of the result images comprise respective values, each value representing an individual test result of a corresponding product.

5 . The method of claim 1 , wherein:
the making the first determination is based on a first feature extracted from the first result images by a neural network; and the making the second determination is based on a second feature extracted from the second result images by the neural network or by another neural network.

6 . The method of claim 5 , wherein the neural network comprises a convolution neural network (CNN) or a transformer network.

7 . The method of claim 5 , wherein the making the first determination comprises:
calculating a statistic indicating a distance measure between a distribution of features of a ground truth image corresponding to the first scale level and a distribution of features of the first result images corresponding to the first scale level; and making the first determination by comparing the statistic with a predetermined reference value.

8 . The method of claim 7 , wherein the comparing the statistic with the predetermined reference value comprises
determining that the first result images satisfies a first condition based on determining that the statistic is greater than the predetermined reference value.

9 . The method of claim 7 ,
wherein the ground truth image is derived from ground truth test result data of a prior test performed by the test equipment.

10 . An apparatus for verifying reliability of a test for products performed by test equipment, the apparatus comprising:
one or more processors and a memory storing instructions configured to cause the one or more processors to perform a process comprising:
receiving result images generated from preprocessing of result data of the test, the result data including labels for a plurality of scale levels and the received result images including first result images belonging to a first scale level of the plurality of scale levels and second result images belonging to a second scale level of the plurality of scale levels;
making a first determination, from the first result images, whether the first scale level is normal or abnormal;
making a second determination, from the second result images, whether the second scale level is normal or abnormal; and
determining whether an error occurred in the test based on the first determination and the second determination.

11 . The apparatus of claim 10 , wherein each label includes an identifier of a scale level.

12 . The apparatus of claim 11 , wherein the process further comprises classifying the result images by their corresponding scale levels according to the labels.

13 . The apparatus of claim 12 , wherein the result data comprises device-under-test (dut) maps respectively associated with the labels, wherein each of the dut maps comprises element areas each comprising a value determined by the test equipment during performance of the test.

14 . The apparatus of claim 10 , wherein:
the making the first determination is based on a first feature extracted from the first result images by a neural network; and the making the second determination is based on a second feature extracted from the second result images by the neural network or by another neural network.

15 . The apparatus of claim 14 , wherein the neural network comprises a convolution neural network (CNN) or a transformer network.

16 . The apparatus of claim 14 , wherein the making the first determination comprises:
calculating a statistic indicating a distance measure between a distribution of features of a ground truth image corresponding to the first scale level and a distribution of features of the first result images corresponding to the first scale level; and making the first determination by comparing the statistic with a predetermined reference value.

17 . The apparatus of claim 16 , wherein the comparing the statistic with the predetermined reference value comprises:
determining that the first result images satisfies a first condition based on determining that the statistic is greater than the predetermined reference value.

18 . The apparatus of claim 16 , wherein
the ground truth image is derived from ground test result data of a prior test performed by the test equipment.

19 . A system for verifying reliability of a test of products, the system comprising:
test computers each configured to perform the test on a respective set of the products and each test computer collecting a corresponding result map having element areas indicating individual test results respectively corresponding products; and a reliability verification computer configured to determine whether a reliability condition of the test is satisfied based on the result maps and based on scale levels of the result maps, the scale levels corresponding to hierarchical scales in testing the products.

20 . The system of claim 19 , wherein
the hierarchal scales includes at least two of: a lot-level scale representing a production block of the products, a board-level scale of boards on which the products are installed for the test, a slot-level scale for the boards to interface with the test computers, and an equipment-level scale of the test computers performing the test on the products installed on the boards.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Kim Keunseo
• Lee Jaecheol
• Park Sangkyu
• Park Sangdo
• Kim Dokyoung
• Yang Minkyu
• Lee Young
• Jang Jinsic

Classifications

• G06F11/3072

  where the reporting involves data filtering, e.g. pattern matching, time or event triggered, adaptive or policy-based reporting · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06V10/20

  Image preprocessing · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/40

  Extraction of image or video features · CPC title