Learning ordinal representations for deep reinforcement learning based object localization

Patent metadata

Field	Value
Publication number	US-12205357-B2
Application number	US-202217715901-A
Country	US
Kind code	B2
Filing date	Apr 7, 2022
Priority date	Apr 8, 2021
Publication date	Jan 21, 2025
Grant date	Jan 21, 2025

Abstract

A reinforcement learning based approach to the problem of query object localization, where an agent is trained to localize objects of interest specified by a small exemplary set. We learn a transferable reward signal formulated using the exemplary set by ordinal metric learning. It enables test-time policy adaptation to new environments where the reward signals are not readily available, and thus outperforms fine-tuning approaches that are limited to annotated images. In addition, the transferable reward allows repurposing of the trained agent for new tasks, such as annotation refinement, or selective localization from multiple common objects across a set of images. Experiments on corrupted MNIST dataset and CU-Birds dataset demonstrate the effectiveness of our approach.

First claim

The invention claimed is:

1. A deep reinforcement learning (RL) method for object localization comprising:
acquiring a seed dataset including a set of seed images each with ground truth bounding box annotation;
pretrain ordinal embedding by randomly perturbing the ground truth bounding box at different levels denoted by parameter p, said ordinal embedding satisfying an ordinal constraint locally for each pair of perturbed data augmented from the same image, wherein the pretraining is performed through the effect of a backbone network, a region of interest (RoI) head, and a triplet loss; and
using an embedding function, configuring RL agents to start from a whole image and recursively sample actions from a discrete action space such that rewards are produced, the rewards of a sample action determined from embedding distances and updating a policy network based on the rewards so determined; and
outputting an annotation policy and embedding function.

2. The method of claim 1 wherein the seed image bounding box annotation is initially provided by a human action.

Assignees

• Nec Lab America Inc
• Nec Corp

Inventors

• Han Shaobo
• Min Renqiang
• Li Tingfeng

Classifications

• G06V30/19167

  Active pattern learning · CPC title

• G06V10/82 Primary

  using neural networks · CPC title

• G06V10/7788 Primary

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title