Systems and methods for image feature extraction

Patent metadata

Field	Value
Publication number	US-11615612-B2
Application number	US-202017112096-A
Country	US
Kind code	B2
Filing date	Dec 4, 2020
Priority date	Dec 4, 2020
Publication date	Mar 28, 2023
Grant date	Mar 28, 2023

Abstract

This description relates to image feature extraction. In some examples, a system can include a keypoint detector and a feature list generator. The keypoint detector can be configured to upsample a keypoint score map to produce an upsampled keypoint score map. The keypoint score map can include feature scores indicative of a likelihood of at least one feature being present at keypoints in an image. The feature list generator can be configured to identify a subset of keypoints of the keypoints in the image using the feature scores of the up sampled keypoint score map, determine descriptors for the subset of keypoints based on a feature description map, and generate a keypoint descriptor map for the image based on the determined descriptors.

First claim

What is claimed is:

1. A system comprising:
a keypoint detector configured to upsample a keypoint score map to produce an upsampled keypoint score map, the keypoint score map comprising feature scores indicative of a likelihood of at least one feature being present at keypoints in an image; and
a feature list generator configured to:
identify a subset of keypoints of the keypoints in the image using the feature scores of the upsampled keypoint score map;
determine descriptors for the subset of keypoints based on a feature description map;
generate a keypoint descriptor map for the image based on the determined descriptors; and
apply a factorization window to the keypoint descriptor map to produce an upsampled keypoint descriptor map.

2. The system of claim 1 , wherein the feature description map is provided by a decoder neural network, the decoder neural network comprising an input layer, an output layer, and intermediate layers between the input layer and the output layer, wherein one of the intermediate layers is configured to provide the feature description map.

3. The system of claim 1 , wherein the feature list generator is further configured to apply a nearest neighbor resize process to the keypoint descriptor map to produce the upsampled keypoint descriptor map, the upsampled keypoint descriptor map having an image resolution that is similar to an image resolution of the image.

4. The system of claim 1 , wherein the factorization window comprises locations associated with respective convolution weights, and applying the factorization window to the keypoint descriptor map comprises multiplying descriptor values associated with locations of the keypoint descriptor map with the respective convolution weights associated with the locations of the factorization window to produce the upsampled keypoint descriptor map.

5. The system of claim 1 , wherein the keypoint detector is further configured to:
resize the keypoint score map from a first image resolution to a second image resolution; and
convolve the resized keypoint score map with at least one filter to provide the upsampled keypoint score map, the upsampled keypoint score map having an image resolution that is similar to an image resolution of the image.

6. The system of claim 5 , wherein the keypoint detector is configured to resize the keypoint score map by implementing a nearest neighbor image scaling.

7. The system of claim 1 , wherein the feature list generator is further configured to:
evaluate the feature scores relative to a feature score threshold to identify a subset of scores;
apply a non-maxima suppression to the subset of scores to identify a group of scores from the subset of scores; and
identify the subset of keypoints of the keypoints in the image based on the group of scores.

8. The system of claim 1 , wherein the feature list generator is further configured to generate keypoint list data based on the upsampled keypoint score map and the upsampled keypoint descriptor map.

9. A method comprising:
upsampling a keypoint score map to produce an upsampled keypoint score map, the keypoint score map comprising feature scores indicative of a likelihood of at least one feature being present at keypoints in an image;
identifying a subset of keypoints of the keypoints in the image using the feature scores of the upsampled keypoint score map;
determining descriptors for the subset of keypoints based on a feature description map;
generating a keypoint descriptor map for the image based on the determined descriptors; and
applying a factorization window to the keypoint descriptor map to produce an upsampled keypoint descriptor map.

10. The method of claim 9 , further comprising receiving from a decoder neural network the feature description map, the decoder neural network comprising an input layer, an output layer, and intermediate layers, wherein one of the intermediate layers is configured to provide the feature description map.

11. The method of claim 9 , further comprising applying a nearest neighbor resize process to the keypoint descriptor map to produce the upsampled keypoint descriptor map, the upsampled keypoint descriptor map having an image resolution that is similar to an image resolution of the image.

12. The method of claim 9 , wherein applying the factorization window to the keypoint descriptor map comprises multiplying descriptor values associated with locations of the keypoint descriptor map with respective convolution weights associated with the locations of the factorization window to produce the upsampled keypoint descriptor map.

13. The method of claim 9 , further comprising:
resizing the keypoint score map from a first image resolution to a second image resolution by implementing a nearest neighbor image scaling;
convolving the resized keypoint score map with at least one filter to provide the upsampled keypoint score map, the upsampled keypoint score map having an image resolution that is similar to an image resolution of the image; and
generating keypoint list data based on the upsampled keypoint score map and the upsampled keypoint descriptor map.

14. The method of claim 9 , further comprising:
evaluating the feature scores relative to a feature score threshold to identify a subset of scores;
applying a non-maxima suppression to the subset of scores to identify a group of scores from the subset of scores; and
identifying the subset of keypoints of the keypoints in the image based on the group of scores.

15. One or more non-transitory computer-readable media having machine readable instructions executable by a processor, the machine readable instructions comprising:
a keypoint detector programmed to upsample a keypoint score map to produce an upsampled keypoint score map, the keypoint score map comprising feature scores indicative of a likelihood of at least one feature being present at keypoints in an image; and
a feature list generator comprising:
a keypoint selector programmed to identify a subset of keypoints of the keypoints in the image using the feature scores of the upsampled keypoint score map; and
a descriptor calculator programmed to:
determine descriptors for the subset of keypoints based on a feature description map and generate a keypoint descriptor map for the image based on the determined descriptors; and
apply a factorization window to the keypoint descriptor map to provide an upsampled keypoint descriptor map.

16. The one or more non-transitory computer-readable media of claim 15 , wherein the descriptor calculator is further programmed to receive from a decoder neural network the feature description map, the decoder neural network comprising an input layer, an output layer, and intermediate layers, wherein one of the intermediate layers is programmed to provide the feature description map.

17. The one or more non-transitory computer-readable media of claim 15 ,
wherein the descriptor calculator is further programmed to apply a nearest neighbor resize process to the keypoint descriptor map to produce the upsampled keypoint descriptor map, the upsampled keypoint descriptor map having an image resolution that is similar to an image resolution of the image, and
wherein applying the factorization window to the keypoint descriptor map by multiplying descriptor values associated with locations of the keypoint descriptor map with respective convolution weights associated with the locations of the factorization window to produce the upsampled keypoint descriptor map.

18. The one or more non-transitory computer-readable media o

Assignees

• Texas Instruments Inc

Inventors

• Poddar Deepak Kumar
• Nagori Soyeb
• Garud Hrushikesh Tukaram
• Desappan Kumar

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06T2207/20084

  Artificial neural networks [ANN] · CPC title

• G06T3/4046

  using neural networks · CPC title

• G06T2207/30252

  Vehicle exterior; Vicinity of vehicle · CPC title