Feature extraction with keypoint resampling and fusion (KRF)

Patent metadata

Field	Value
Publication number	US-11676018-B2
Application number	US-202117162246-A
Country	US
Kind code	B2
Filing date	Jan 29, 2021
Priority date	Jan 30, 2020
Publication date	Jun 13, 2023
Grant date	Jun 13, 2023

Abstract

A method of feature extraction from an image can include receiving the image including pixels, generating confidence values corresponding to positions of the pixels in the image by an artificial intelligence (AI) based feature extractor, selecting a first position among the positions of the pixels in the image, a first confidence value among the generated confidence values at the first position being higher than a first threshold, and generating a final set of keypoint-descriptor pairs based on the confidence values corresponding to positions of the pixels in the image. The final set of keypoint-descriptor pairs includes at least two keypoint-descriptor pairs corresponding to the first position among the positions of the pixels in the image.

First claim

What is claimed is:

1. A method of feature extraction from an image, comprising:
receiving the image including pixels;
generating confidence values corresponding to positions of the pixels in the image by an artificial intelligence based feature extractor;
selecting a first position among the positions of the pixels in the image, a first confidence value among the generated confidence values at the first position being higher than a first threshold; and
generating a final set of keypoint-descriptor pairs based on the confidence values corresponding to the positions of the pixels in the image, the final set of keypoint-descriptor pairs including at least two keypoint-descriptor pairs corresponding to the first position among the positions of the pixels in the image.

2. The method of claim 1 , further comprising:
selecting a first set of positions from the positions of the pixels in the image to be a first set of keypoints based on the generated confidence values, each of the first set of the positions corresponding to one of the generated confidence values that is above a second threshold; and
selecting a second set of positions from the positions of the pixels in the image to be a second set of keypoints based on the generated confidence values, each of the second set of positions corresponding to one of the generated confidence values that is above the first threshold, the first set of positions including the second set of positions,
wherein
the selecting the second set of positions from the positions of the pixels in the image includes the selecting of the first position among the positions of the pixels in the image, and
the final set of keypoint-descriptor pairs includes a first set of keypoint-descriptor pairs corresponding to members of the first set of the keypoints and a second set of keypoint-descriptor pairs corresponding to members of the second set of the keypoints.

3. The method of claim 1 , further comprising:
selecting a first set of positions from the positions of the pixels in the image to be a first set of keypoints based on the generated confidence values, each of the first set of the positions corresponding to one of the generated confidence values that is above a second threshold; and
selecting a second set of positions from the first set of positions in the image to be a second set of keypoints based on the generated confidence values, each of the second set of positions corresponding to one of the generated confidence values that is above the first threshold,
wherein
the selecting the second set of positions from the first set of positions in the image includes the selecting of the first position among the positions of the pixels in the image, and
the final set of keypoint-descriptor pairs includes a first set of keypoint-descriptor pairs corresponding to members of the first set of the keypoints and a second set of keypoint-descriptor pairs corresponding to members of the second set of the keypoints.

4. The method of claim 1 , further comprising:
determining whether the first confidence value among the generated confidence values at the first position is higher than the first threshold; and
in response to the first confidence value among the generated confidence values at the first position being higher than the first threshold,
selecting a first keypoint descriptor from the first position in a first descriptor map including keypoint descriptors corresponding to the positions of the pixels in the image, and
selecting a second keypoint descriptor from the first position in a second descriptor map including keypoint descriptors corresponding to the positions of the pixels in the image,

wherein the at least two keypoint-descriptor pairs include two keypoint-descriptor pairs corresponding to the first keypoint descriptor and the second keypoint descriptor.

5. The method of claim 1 , further comprising:
generating a first descriptor map by the artificial intelligence based feature extractor; and
generating a second descriptor map,
wherein the at least two keypoint-descriptor pairs include two keypoint-descriptor pairs corresponding to a first keypoint descriptor in the first descriptor map and a second keypoint descriptor in the second descriptor map.

6. The method of claim 5 , wherein the first keypoint descriptor in the first descriptor map is a first binary vector, and the second keypoint descriptor in the second descriptor map is a second binary vector complementary to the first binary vector, where 1 or 0 in the first binary vector is switched to 0 and 1, respectively, in the second binary vector.

7. The method of claim 1 , further comprising:
resizing the image to generate a resized image:
determining keypoint-descriptor pairs corresponding to the resized image; and
including the keypoint-descriptor pairs corresponding to the resized image in the final set of keypoint-descriptor pairs.

8. An apparatus of feature extraction from an image, comprising circuitry configured to:
receive the image including pixels;
generate confidence values corresponding to positions of the pixels in the image by an artificial intelligence based feature extractor;
select a first position among the positions of the pixels in the image, a first confidence value among the generated confidence values at the first position being higher than a first threshold; and
generate a final set of keypoint-descriptor pairs based on the confidence values corresponding to the positions of the pixels in the image, the final set of keypoint-descriptor pairs including at least two keypoint-descriptor pairs corresponding to the first position among the positions of the pixels in the image.

9. The apparatus of claim 8 , wherein the circuitry is further configured to:
select a first set of positions from the positions of the pixels in the image to be a first set of keypoints based on the generated confidence values, each of the first set of the positions corresponding to one of the generated confidence values that is above a second threshold; and
select a second set of positions from the positions of the pixels in the image to be a second set of keypoints based on the generated confidence values, each of the second set of positions corresponding to one of the generated confidence values that is above the first threshold, the first set of positions including the second set of positions,
wherein
the second set of positions includes the first position among the positions of the pixels in the image, and
the final set of keypoint-descriptor pairs includes a first set of keypoint-descriptor pairs corresponding to members of the first set of the keypoints and a second set of keypoint-descriptor pairs corresponding to members of the second set of the keypoints.

10. The apparatus of claim 8 , wherein the circuitry is further configured to:
select a first set of positions from the positions of the pixels in the image to be a first set of keypoints based on the generated confidence values, each of the first set of the positions corresponding to one of the generated confidence values that is above a second threshold; and
select a second set of positions from the first set of positions in the image to be a second set of keypoints based on the generated confidence values, each of the second set of positions corresponding to one of the generated confidence values that is above the first threshold,
wherein
the second set of positions includes the first position among the positions of the pixels in the image, and
the final set of keypoint-descriptor pairs includes a first set of keypoint-descriptor pairs corresponding to members of the first set of the keypoints and a second set of keypoint-descriptor pa

Assignees

• Mediatek Inc

Inventors

• Lee Chia-Da
• Wong Wai Mun
• Tsung Pei-Kuei

Classifications

• G06T3/40

  Scaling of whole images or parts thereof, e.g. expanding or contracting · CPC title

• G06V10/50

  by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis · CPC title

• G06N20/00

  Machine learning · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06N3/08 Primary

  Learning methods · CPC title