Reducing false negatives and finding new classes in object detectors

Patent metadata

Field	Value
Publication number	US-12165409-B2
Application number	US-202017116421-A
Country	US
Kind code	B2
Filing date	Dec 9, 2020
Priority date	Dec 9, 2020
Publication date	Dec 10, 2024
Grant date	Dec 10, 2024

Abstract

Reducing false negatives and finding new classes in object detectors is disclosed. Also disclosed is a method that includes grouping together a plurality of cropped image portions from region proposals based on image properties. The method also includes receiving first user input that either establishes that the grouped together, cropped image portions relate to uninteresting objects or establishes that the grouped together, cropped image portions relate to an object class of interesting objects. The method also includes obtaining, only when the grouped together, cropped image portions relate to the object class of interesting objects, second user input that includes an object label corresponding to the object class of interesting objects.

First claim

What is claimed is:

1. A method comprising:
receiving video at a first object detector that generates first output data, a portion of the first output data including identification of one or more objects of interest within the video;
analyzing the first output data to make a determination that each of a plurality of factors indicate a potential missed object detection within the video;
sending the video from a system site to a cloud server based on the determination, the cloud server including a second object detector having more computational resources than the first object detector;
inputting the video to the second object detector to generate second output data therefrom, a portion of the second output data identifying a potential interesting object corresponding to the potential missed object detection, and the potential interesting object being more difficult to classify into a respective object class than any of the one or more objects of interest previously identified by the first object detector;
receiving first user input needed for establishing that either the potential interesting object is an uninteresting object or confirms that the potential interesting object is an interesting object; and
only when the potential interesting object is a confirmed interesting object, obtaining second user input that includes an object label corresponding to the confirmed interesting object.

2. The method of claim 1 wherein the factors include existence of a plurality of region proposals, lack of a detected object and background segmentation data.

3. The method of claim 1 wherein each of the first and second object detectors includes at least one convolutional neural network.

4. The method of claim 1 further comprising modifying the second object detector to reflect the object label.

5. The method of claim 1 further comprising modifying the first object detector to reflect the object label.

6. The method of claim 1 further comprising capturing the video at a security video camera, the capturing occurring prior to the receiving of the video at the first object detector.

7. The method of claim 1 wherein the object label is a new object label.

8. A system comprising:
a cloud server;
a system site in communication with the cloud server and including a first object detector configured to receive video and generate first output data therefrom, wherein a portion of the first output data includes identification of one or more objects of interest within the video, and the system site being configured to:
analyze the first output data to make a determination that each of a plurality of factors indicate a potential missed object detection within the video; and
sending the video to the cloud server based on the determination, and

the cloud server including a second object detector having more computational resources than the first object detector, and the second object detector configured to receive the video to generate second output data therefrom, a portion of the second output data identifying a potential interesting object corresponding to the potential missed object detection, and the potential interesting object being more difficult to classify into a respective object class than any of the one or more objects of interest previously identified by the first object detector; and
a client device configured to:
receiving first user input needed for establishing that either the potential interesting object is an uninteresting object or confirms that the potential interesting object is an interesting object; and
only when the potential interesting object is a confirmed interesting object, obtaining second user input that includes an object label corresponding to the confirmed interesting object.

9. The system of claim 8 wherein the factors include existence of a plurality of region proposals, lack of a detected object and background segmentation data.

10. The system of claim 8 wherein each of the first and second object detectors includes at least one convolutional neural network.

11. The system of claim 8 wherein the cloud server is configured to modify the second object detector to reflect the object label.

12. The system of claim 8 the system site is further configured to modify the first object detector to reflect the object label.

13. The system of claim 8 wherein the system site further includes at least one security video camera configured to capture the video prior to the video being received at the first object detector.

14. The system of claim 8 wherein the object label is a new object label.

15. A computer-implemented method comprising:
grouping together, by employing at least one processor, a plurality of cropped image portions from region proposals based on image properties;
employing the at least one processor to determine, based on feature similarity in respect of the cropped image portions, that rather than ignoring the region proposals, instead at least first user input should be obtained;
after a determination that the first user input should be obtained, receiving the first user input that either establishes that the grouped together, cropped image portions relate to uninteresting objects or establishes that the grouped together, cropped image portions relate to an object class of interesting objects; and
only when the grouped together, cropped image portions relate to the object class of interesting objects, obtaining second user input that includes an object label corresponding to the object class of interesting objects.

16. The computer-implemented method of claim 15 wherein the grouping together of the cropped image portions is carried out within a system site that includes at least one client device.

17. The computer-implemented method of claim 16 wherein the client device carries out the receiving of the first user input and the obtaining of the second user input.

18. The computer-implemented method of claim 17 wherein the client device is a mobile device.

19. The computer-implemented method of claim 15 wherein the object class is a new object class.

20. The computer-implemented method of claim 15 wherein the grouping together of the cropped image portions is carried out within a cloud server.

Assignees

• Motorola Solutions Inc

Inventors

• Anantha Aravind
• Verelli Jason
• Beaulieu Patrick
• Booth Daniel

Classifications

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V20/49

  Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes · CPC title

• G06F16/45

  Clustering; Classification · CPC title

• G06V10/7788

  the supervisor being a human, e.g. interactive learning with a human teacher · CPC title