Devices, systems, and methods for detecting unknown objects

Patent metadata

Field	Value
Publication number	US-10867216-B2
Application number	US-201715457067-A
Country	US
Kind code	B2
Filing date	Mar 13, 2017
Priority date	Mar 15, 2016
Publication date	Dec 15, 2020
Grant date	Dec 15, 2020

Abstract

Devices, systems, and methods obtain a region of an image; generate known-object scores for the region using known-object detectors, wherein each known-object detector of the known-object detectors detects objects in a respective object class; determine a likelihood that the region includes a complete object; and determine a likelihood that the region includes an unknown object based on the likelihood that the region includes a complete object and on the known-object scores.

First claim

What is claimed is:

1. A system comprising:
one or more computer-readable media; and
one or more processors that are coupled to the one or more computer-readable media and that are configured to cause the system to
generate respective known-object probabilities that a region of an image includes a known-object in a plurality of object classes using a plurality of known-object detectors, wherein each known-object detector of the plurality of known-object detectors detects objects in a respective object class of the plurality of object classes;
generate an unknown-object probability that the region includes an unknown object using the plurality of known-object detectors;
determine whether the region includes an unknown object based on the respective known-object probabilities for the plurality of object classes and the unknown-object probability;
create a new class that includes the unknown object, if the region is determined to include the unknown object;
search for a labeled image which matches the unknown object;
assign a label of the matched labeled image to the unknown object as a new known-object label of the created new class; and
generate a new known-object detector for the created new class.

2. The system of claim 1 , wherein the known-object detectors each include a respective scoring function that detects a respective known object.

3. The system of claim 2 , wherein the one or more processors are further configured to cause the system to
for each known-object detector, determine a respective known-object probability that is a probability that the region includes an object in the respective object class that is detected by the known-object detector, thereby generating known-object probabilities.

4. The system of claim 3 , wherein, to determine a respective known-object probability, the one or more processors are configured to cause the system to perform operations that can be described by

P
⁡

(

C
i

|
x

)

=

exp
⁡

(

S
⁡

(

C
i

,
x

)

)

Σ

k
=
1

K

⁢

exp
⁡

(

S
⁡

(

C
k

,
x

)

)

,

where x is the image region, where S(C i ,x) is a scoring function of a known-object detector that detects objects in object class C i , where S(C k , x) is a scoring function of a known-object detector that detects objects in object class C k , where K is a total number of known-object detectors, and where P(C i |x) is the known-object probability for object class C.

5. A method comprising:
generating respective known-object probabilities that a region of an image includes a known object in a plurality of object classes using a plurality of known-object detectors, wherein each known-object detector of the plurality of known-object detectors detects objects in a respective object class of the plurality of object classes;
generating an unknown-object probability that the region includes an unknown object using the plurality of known-object detectors;
determining whether the region includes an unknown object based on the respective known-object probabilities for the plurality of object classes and the unknown-object probability;
creating a new class that includes the unknown object, if the region is determined to include the unknown object;
searching for a labeled image which matches the unknown object;
assigning a label of the matched labeled image to the unknown object as a new known-object label of the created new class; and
generating a new known-object detector for the created new class.

6. One or more computer-readable media storing instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
generating respective known-object probabilities that a region of an image includes a known object in a plurality of object classes using a plurality of known-object detectors, wherein each known-object detector of the plurality of known-object detectors detects objects in a respective object class of the plurality of object classes;
generating an unknown-object probability that the region includes an unknown object using the plurality of known-object detectors;
determining whether the region includes an unknown object based on the respective known-object probabilities for the plurality of object classes and the unknown-object probability;
creating a new class that includes the unknown object, if the region is determined to include the unknown object;
searching for a labeled image which matches the unknown object;
assigning a label of the matched labeled image to the unknown object as a new known-object label of the created new class; and
generating a new known-object detector for the created new class.

7. The one or more computer-readable media of claim 6 , wherein the known-object detectors each include a respective classifier.

Assignees

• Canon Kk

Inventors

• Skaff Sandra
• Yu Jie
• Imai Francisco

Classifications

• G06V10/759

  Region-based matching · CPC title

• G06V20/00 Primary

  Scenes; Scene-specific elements (control of digital cameras H04N23/60) · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• G06F18/2415

  based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate · CPC title

• G06F18/2433

  Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection · CPC title