Pointer recognition for analog instrument image analysis

Patent metadata

Field	Value
Publication number	US-11301712-B2
Application number	US-201916677481-A
Country	US
Kind code	B2
Filing date	Nov 7, 2019
Priority date	Nov 7, 2019
Publication date	Apr 12, 2022
Grant date	Apr 12, 2022

Abstract

Systems and processes for identifying a pointer in an image of an analog instrument are provided herein. An instrument contour in the image corresponding to the analog instrument may be identified. A plurality of candidate pointer contours in the image may be identified and screened using one or more geometric property screening techniques including an evaluation of a geometric area, a distance parameter, and/or a gravity center of the plurality of candidate pointer contours. Principal component analysis (PCA) may be performed to select an identified pointer contour from among the reduced plurality of candidate pointer contours. A linear regression model may be applied to pixel points in the contour area of the identified pointer contour and a slope and angle of an associated pointer represented by the identified pointer contour may be determined based on an output of the linear regression model.

First claim

We claim:

1. A method for identifying a pointer in an image of an analog instrument, the method comprising:
identifying an instrument contour in the image corresponding to the analog instrument;
identifying a plurality of candidate pointer contours in the image;
screening the plurality of candidate pointer contours in the image using one or more geometric property screening techniques, the one or more geometric property screening techniques being based on an evaluation of a geometric area, a distance parameter, and/or a gravity center of the plurality of candidate pointer contours; and
responsive to determining that the number of candidate pointer contours remaining after the screening is greater than one, performing principal component analysis (PCA) to select an identified pointer contour from among the reduced plurality of candidate pointer contours, the identified pointer contour having the greatest contribution rate of a first principal component of the reduced plurality of candidate pointer contours.

2. The method of claim 1 , wherein the one or more geometric property screening techniques includes identifying a subset of the plurality of candidate pointer contours based on the geometric area of each of the plurality of candidate pointer contours, each of the pointer contours in the subset having a respective geometric area that is between a selected minimum area and a selected maximum area.

3. The method of claim 2 , wherein the selected minimum area and the selected maximum area are selected to define a corresponding area range that is a fraction of an area of the instrument contour.

4. The method of claim 1 , wherein the one or more geometric property screening techniques includes identifying a subset of the plurality of candidate pointer contours based on a distance parameter of each of the plurality of candidate pointer contours, each of the pointer contours in the subset having a distance between a respective gravity center of the pointer contour and a geometric center of the instrument contour that is less than a selected threshold distance.

5. The method of claim 1 , wherein the one or more geometric property screening techniques includes identifying a subset of the plurality of candidate pointer contours based on a gravity center location of each of the plurality of candidate pointer contours, each of the pointer contours in the subset having a gravity center that is inside of the respective pointer contour.

6. The method of claim 1 , wherein the image is a binary-value image of the analog instrument.

7. The method of claim 1 , further comprising applying a linear regression model to pixel points in the contour area of the identified pointer contour and determining a slope and angle of an associated pointer represented by the identified pointer contour based on an output of the linear regression model.

8. The method of claim 7 , wherein the linear regression model is a machine learning model trained by the pixel points in the contour area of the identified pointer contour.

9. The method of claim 7 , further comprising determining an analog instrument reading based on the determined slope and angle of the identified pointer contour and providing the analog instrument reading to a selected system.

10. The method of claim 9 , wherein the selected system comprises a machine learning system and/or a connected device, the method further comprising controlling the connected device based on the analog instrument reading.

11. A system for pointer recognition in an image of an analog instrument, the system comprising:
one or more memories;
one or more processing units coupled to the one or more memories;
one or more computer readable storage media storing instructions that, when loaded into the one or more memories, cause the one or more processing units to perform image analysis operations for:
identifying an instrument contour in the image corresponding to the analog instrument;
identifying a plurality of candidate pointer contours in the image;
screening the plurality of candidate pointer contours in the image using at least one geometric property screening technique to generate a reduced subset of candidate pointer contours, the at least one geometric property screening technique being based on an evaluation of a geometric area, a distance parameter, and/or a gravity center of the plurality of candidate pointer contours;
selecting an identified pointer contour from among the candidate pointer contours in the reduced subset, where the identified pointer contour is selected as the candidate pointer contour in the reduced subset when the reduced subset includes a single candidate pointer contour, and where the identified pointer contour is selected by performing principal component analysis (PCA) when the reduced subset includes more than one candidate pointer contours, the identified pointer contour selected by performing PCA having the greatest contribution rate of a first principal component of the candidate pointer contours in the reduced subset; and
applying a linear regression model to pixel points in the contour area of the identified pointer contour and determining a slope and angle of an associated pointer represented by the identified pointer contour based on an output of the linear regression model.

12. The system of claim 11 , wherein the evaluation of the geometric area includes comparing a geometric area of each of the plurality of candidate pointer contours to a selected minimum area and a selected maximum area, each of the candidate pointer contours in the reduced subset having a respective geometric area that is between a selected minimum area and a selected maximum area.

13. The system of claim 12 , wherein the selected minimum area and the selected maximum area are selected to define a corresponding area range that is a fraction of an area of the instrument contour.

14. The system of claim 11 , wherein the evaluation of the distance parameter includes comparing a distance between a respective gravity center of each candidate pointer contour and a geometric center of the instrument contour to a selected threshold distance, and wherein each of the candidate pointer contours in the reduced subset have a distance between the respective gravity center of the candidate pointer contour and the geometric center of the instrument contour that is less than the selected threshold distance.

15. The system of claim 11 , wherein the evaluation of the gravity center includes determining a location of the gravity center of each candidate pointer contour relative to boundaries of the respective candidate pointer contour, each of the candidate pointer contours in the reduced subset having a respective gravity center location that is inside of the respective candidate pointer contour.

16. The system of claim 11 , wherein the linear regression model is a machine learning model trained by the pixel points in the contour area of the identified pointer contour.

17. The system of claim 11 , wherein an analog instrument reading based on the determined slope and angle of the identified pointer contour is provided as input to a machine learning system, and wherein the input analog instrument reading is configured to train the machine learning system and/or to trigger output of a result of a machine learning operation of the machine learning system, and wherein the result of the machine learning operation of the machine learning system includes a control signal to control the system and/or a connected device based on the determined slope and angle of the identified pointer contour.

Assignees

• Sap Se

Inventors

• He Jie
• Li Yanbing
• Liu Hong
• Lou Yubo
• Cai Lin
• Wang Xuemin

Classifications

• G06V10/44 Primary

  Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components · CPC title

• G06V10/7715

  Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods · CPC title

• G06F18/2135

  based on approximation criteria, e.g. principal component analysis · CPC title

• G06T2207/20081

  Training; Learning · CPC title

• G06T7/73

  using feature-based methods · CPC title