Accelerated non-maximum suppression in machine learning applications

Patent metadata

Field	Value
Publication number	US-11989948-B1
Application number	US-202117500167-A
Country	US
Kind code	B1
Filing date	Oct 13, 2021
Priority date	Oct 13, 2021
Publication date	May 21, 2024
Grant date	May 21, 2024

Abstract

Apparatuses, systems, and techniques to perform non-maximum suppression (NMS) with a bit-reduced radix sort to remove redundant bounding boxes are described. In at least one embodiment, one or more circuits perform i) a bit-reduced radix sort operation to sort a list of confidence scores associated with a set of bounding boxes corresponding to one or more objects within one or more digital images and ii) a non-maximum suppression (NMS) operation on the sorted list to remove one or more redundant bounding boxes from the set.

First claim

What is claimed is:

1. A method of performing a non-maximum suppression (NMS) algorithm, the method comprising:
identifying a plurality of bounding boxes corresponding to one or more objects associated with one or more digital images, wherein each of the plurality of bounding boxes is associated with a confidence score in a first set of confidence scores;
transforming the first set of confidence scores into a second set of confidence scores within a specified interval, wherein a first portion of each confidence score in the second set of confidence scores is the same;
sorting the confidence scores in the second set of confidence scores in a descending order according to a remaining portion of each confidence score in the second set of confidence scores; and
performing a first suppression operation on the second set of confidence scores to remove one or more redundant bounding boxes from the plurality of bounding boxes.

2. The method of claim 1 , wherein each of the second set of confidence scores comprises the first portion, a second portion, and a third portion, wherein the first portion of each of the second set of confidence scores is the same and the second portion of each of the second set of confidence scores is the same.

3. The method of claim 1 , wherein each of the first set of confidence scores is represented as a floating-point number according to the IEEE 754 format, wherein the floating-point number comprises a sign bit, a biased-exponent field, and a mantissa field, wherein each of the second set of confidence scores comprises same values for the sign bit and the biased-exponent field after the transforming.

4. The method of claim 3 , wherein sorting the confidence scores in the second set of confidence scores comprises executing an argsort function on only mantissa bits of the mantissa fields of the second set of confidence scores.

5. The method of claim 3 , wherein sorting the confidence scores in the second set of confidence scores comprises executing an argsort function on only a portion of mantissa bits of the mantissa fields of the second set of confidence scores.

6. The method of claim 3 , wherein each of the first set of confidence scores is in an interval between zero and one, wherein the specified interval is between one and two, which is clipped by a number of mantissa bits in the mantissa field.

7. The method of claim 1 , wherein each of the first set of confidence scores is in an interval between zero and one, wherein the specified interval is a half-open interval between one and two but excluding two.

8. The method of claim 1 , wherein the second set of confidence scores is part of a plurality of classes, and wherein the first suppression operation comprises selecting a specified number of bounding boxes having the highest confidence scores in the second set of confidence scores.

9. The method of claim 1 , wherein the first set of confidence scores is part of a first class of objects, and wherein the method further comprises:
identifying a second plurality of bounding boxes corresponding to one or more objects associated with the one or more digital images, wherein each of the second plurality of bounding boxes is associated with a confidence score in a third set of confidence scores, wherein the third set of confidence scores is part of a second class of objects;
transforming the third set of confidence scores into a fourth set of confidence scores within the specified interval, wherein a first portion of each confidence score in the fourth set of confidence scores is the same;
sorting the confidence scores in the fourth set of confidence scores in a descending order according to a remaining portion of each confidence score in the fourth set of confidence scores; and
performing a second suppression operation on the fourth set of confidence scores to remove one or more redundant bounding boxes from the second plurality of bounding boxes.

10. The method of claim 9 , further comprising:
combining the remaining bounding boxes associated with the second set of confidence scores and the fourth set of confidence scores into a third plurality of bounding boxes, wherein each of the third plurality of bounding boxes is associated with a confidence score in a fifth set of confidence scores;
sorting the confidence scores in the fifth set of confidence scores in a descending order according to a remaining portion of each confidence score in the fifth set of confidence scores; and
performing a third suppression operation on the fourth set of confidence scores to remove one or more redundant bounding boxes from the second plurality of bounding boxes.

11. A system comprising:
a memory device; and
a processing unit coupled to the memory device, the processing unit to:
identify a plurality of bounding boxes corresponding to one or more objects associated with one or more digital images, wherein each of the plurality of bounding boxes is associated with a confidence score in a first set of confidence scores;
transform the first set of confidence scores into a second set of confidence scores within a specified interval, wherein a first portion of each confidence score in the second set of confidence scores is the same;
sort the confidence scores of the second set of confidence scores in a descending order according to a remaining portion of each confidence score in the second set of confidence scores; and
perform a first suppression operation on the second set of confidence scores in the descending order to remove one or more redundant bounding boxes from the plurality of bounding boxes.

12. The system of claim 11 , wherein each of the second set of confidence scores comprises the first portion, a second portion, and a third portion, wherein the first portion of each of the second set of confidence scores is the same and the second portion of each of the second set of confidence scores is the same.

13. The system of claim 11 , wherein each of the first set of confidence scores is represented as a floating-point number according to the IEEE 754 format, wherein the floating-point number comprises a sign bit, a biased-exponent field, and a mantissa field, wherein each of the second set of confidence scores comprises same values for the sign bit and the biased-exponent field after the transforming.

14. The system of claim 13 , wherein the processing unit is to sort the confidence scores in the second set of confidence scores by executing an argsort function on only mantissa bits of the mantissa fields of the second set of confidence scores.

15. The system of claim 13 , wherein the processing unit is to sort the confidence scores in the second set of confidence scores by executing an argsort function on only a portion of mantissa bits of the mantissa fields of the second set of confidence scores.

16. The system of claim 13 , wherein each of the first set of confidence scores is in an interval between zero and one, wherein the specified interval is between one and two, which is clipped by a number of mantissa bits in the mantissa field.

17. The system of claim 11 , wherein the second set of confidence scores is part of a plurality of classes, and wherein the first suppression operation comprises selecting a specified number of bounding boxes having the highest confidence scores in the second set of confidence scores.

18. The system of claim 11 , wherein the first set of confidence scores is part of a first class of objects, and wherein the processing unit is further to:
identify a second plurality of bounding boxes corresponding to one or more

Assignees

• Nvidia Corp

Inventors

• Fan Zhimeng

Classifications

• G06V20/58 Primary

  Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F7/24

  Sorting, i.e. extracting data from one or more carriers, rearranging the data in numerical or other ordered sequence, and rerecording the sorted data on the original carrier or on a different carrier or set of carriers {sorting methods in general}(G06F7/36 takes precedence) · CPC title