Methods and systems for processing an image

Patent metadata

Field	Value
Publication number	US-12093779-B2
Application number	US-202318235443-A
Country	US
Kind code	B2
Filing date	Aug 18, 2023
Priority date	Oct 15, 2018
Publication date	Sep 17, 2024
Grant date	Sep 17, 2024

Abstract

A system performs a method for processing an image of a machine-readable code. The method includes receiving an image of a machine-readable code comprising coded information, where the machine-readable code is at least partially obscured. An adjusted image is generated by adjusting a color space of the image. At least a machine-readable code region of the image is binarized, wherein the machine-readable code region of the image depicts the machine-readable code. The binarized machine-readable code region is decoded to determine the coded information. Other apparatus and methods are also described.

First claim

The invention claimed is:

1. A computer-implemented method for processing an image of a machine-readable code affixed to an item of a surgical system, the surgical system including a display and one or more processors, the method comprising:
accessing, by the one or more processors, the image that depicts the machine-readable code;
determining, with the one or more processors, at least one corner of the machine-readable code is occluded, false, or blurry such that a machine-readable code region of the image is not locatable;
providing, with the one or more processors, the image as an input to a trained neural network;
receiving, with the one or more processors and as an output from the trained neural network, a cleaned image of the machine-readable code;
locating, with the one or more processors, the machine-readable code region of the cleaned image of the machine-readable code;
decoding, with the one or more processors, coded information contained within the machine-readable code region of the cleaned image;
outputting, with the one or more processors, the decoded information; and
displaying, on the display, patient information generated by the surgical system based on the decoded information.

2. The method of claim 1 , wherein the step of locating the machine-readable code region further comprises performing at least one of corner detection or edge detection by:
applying a corner detection algorithm to the cleaned image to generate a heatmap including heatmap values corresponding to each coordinate within the image; and
detecting the corners of the machine-readable code in the cleaned image based on the heatmap generated by the corner detect algorithm.

3. The method of claim 1 , further comprising, with the trained neural network:
identifying a set of coordinates in the image as likely corresponding to the machine-readable code;
identifying at least one outlier coordinate of the set of coordinates that is unlikely to correspond to the machine-readable code; and
removing the at least one outlier coordinate to generate a cleaned set of coordinates.

4. The method of claim 3 , wherein the step of decoding the coded information further comprises:
identifying the cleaned set of coordinates; and
analyzing a portion of the cleaned image which corresponds to the cleaned set of coordinates.

5. The method of claim 1 , further comprising decreasing high frequency noise of the image to generate the cleaned image.

6. The method of claim 5 , further comprising applying at least one smoothing algorithm to decrease the high frequency noise of the image.

7. The method of claim 5 , further comprising applying a bilateral filter on the image to decrease the high frequency noise of the image while preserving high frequency signals.

8. The method of claim 1 , further comprising increasing a signal to noise ratio of the image to generate the cleaned image.

9. The method of claim 8 , further comprising applying at least one deblurring algorithm to the image to increase the signal to noise ratio.

10. The method of claim 1 , wherein the step of determining that at least one corner of the machine-readable code is occluded, false, or blurry further comprises:
applying a corner detection algorithm to the image; and
identifying the at least one corner of the machine-readable code as occluded, false, or blurry by analyzing an output of the corner detection algorithm.

11. The method of claim 1 , wherein the trained neural network is trained on manually cleaned images of machine-readable code and/or images of obscured machine-readable codes.

12. A computer-implemented method for processing an image of a machine-readable code affixed to an item of a surgical system, the surgical system including a display and one or more processors, the method comprising:
accessing, by one or more processors, an image that depicts a machine-readable code;
determining, by the one or more processors, that at least one corner of the machine-readable code is occluded, false, or blurry;
providing, by the one or more processors, the image as an input to a trained neural network;
receiving, by the one or more processors and as an output from the trained neural network, a cleaned image of the machine-readable code;
applying a corner detection algorithm to the cleaned image to generate a heatmap including heatmap values corresponding to each coordinate within the image;
detecting the corners of the machine-readable code in the cleaned image based on the heatmap generated by the corner detection algorithm;
decoding, with the one or more processors, coded information contained within the machine-readable code of the cleaned image;
outputting, with the one or more processors, the decoded information; and
displaying, on the display, patient information generated by the surgical system based on the decoded information.

13. The method of claim 12 , further comprising decreasing high frequency noise of the image to generate the cleaned image.

14. The method of claim 12 , further comprising increasing a signal to noise ratio of the image to generate the cleaned image.

15. The method of claim 12 , further comprising, with the trained neural network:
identifying a set of coordinates in the image as likely corresponding to the machine-readable code;
identifying outlier coordinates of the set of coordinates that are unlikely to correspond to the machine-readable code; and
removing the outlier coordinates to generate a cleaned set of coordinates.

16. A computer-implemented method for processing an image of a machine-readable code affixed to an item of a surgical system, the surgical system including a display and one or more processors, the method comprising:
accessing, by one or more processors, an image that depicts a machine-readable code;
determining, by the one or more processors, that at least one corner of the machine-readable code is occluded, false, or blurry;
providing, by the one or more processors, the image as an input to a trained neural network;
processing, with the trained neural network, at least a region of the image to generate a cleaned image by:
identifying a set of coordinates in the image that likely correspond to the machine-readable code;
removing outlier coordinates from the set of coordinates to generate a cleaned set of coordinates which correspond to an estimated position of the machine-readable code;

receiving, by the one or more processors and as an output from the trained neural network, the cleaned image of the machine-readable code;
decoding, with the one or more processors, coded information contained within the machine-readable code of the cleaned image;
outputting, with the one or more processors, the decoded information; and
displaying, on the display, patient information generated by the surgical system based on the decoded information.

17. The method of claim 16 , wherein the step of decoding the coded information further comprises:
identifying the cleaned set of coordinates; and
analyzing a portion of the cleaned image which corresponds to the cleaned set of coordinates.

18. The method of claim 16 , further comprising decreasing high frequency noise of the image to generate the cleaned image.

19. The method of claim 18 , further comprising applying at least one smoothing algorithm to decrease the high frequency noise of the image.

20. The method of claim 16 , wherein the step of locating the machine-readable code further comprises performing at least one of corner detection or edge de

Assignees

• Gauss Surgical Inc

Inventors

• Satish Siddarth
• Miller Kevin J

Classifications

• G06K19/0614

  the marking being selective to wavelength, e.g. color barcode or barcodes only visible under UV or IR (methods or arrangements for sensing record carriers using a selected wavelength, see G06K7/12) · CPC title

• G06K19/06037

  multi-dimensional coding · CPC title

• G06K19/027

  the material being suitable for use as a textile, e.g. woven-based RFID-like labels designed for attachment to laundry items (markings attached to laundry items in general D06F93/00) · CPC title

• G06K7/1452

  detecting bar code edges · CPC title

• G06T7/13

  Edge detection · CPC title