Intelligent insect trap and monitoring system

Patent metadata

Field	Value
Publication number	US-12458010-B2
Application number	US-202217742350-A
Country	US
Kind code	B2
Filing date	May 11, 2022
Priority date	May 11, 2021
Publication date	Nov 4, 2025
Grant date	Nov 4, 2025

Abstract

An intelligent insect trap and identification system is disclosed. The intelligent insect trap and identification system can include an insect imaging chamber and identification system. The chamber can include a first cell for accepting insects, a second cell, a first reflector in the second cell, and a first imaging device in the second cell for recording one or more first visual images of the one or more insects in the first cell. Based on the image, the insect imaging chamber can detect and identify the insects. Other aspects, embodiments, and features are also claimed and described.

First claim

What is claimed is:

1 . A method for insect classification, comprising:
receiving a set of a plurality of images including one or more objects; generating one or more bounding boxes around a subset of the one or more objects in an image of the set of the plurality of images, the one or more bounding boxes corresponding to a subset one or more objects; generating an N-frame history buffer for each of the one or more bounding boxes in the image; determining validity of each of the one or more bounding boxes in the image based on the N-frame history buffer of a respective bounding box of the one or more bounding boxes; in response to the validity of each of the one or more bounding boxes in the image, selecting a subset of the one or more bounding boxes in the image; transmitting one or more final bounding boxes based on the subset of the one or more bounding boxes in the image to a deep learning model; receiving, from the deep learning model, one or more classifications of the one or more final bounding boxes; identifying a predetermined second window including the image; identifying a representing size of bounding boxes in the predetermined second window, the bounding boxes in the predetermined second window corresponding to each of the subset of the one or more bounding boxes in the image; identifying one or more outlier bounding boxes in the bounding boxes, each of the one or more outlier bounding boxes being a predetermined percentage larger or smaller than the representing size; and in response to a number of valid images in the predetermined second window being equal to or greater than a predetermined value, transmitting the bounding boxes in the predetermined second window to the deep learning model, wherein each of the valid images includes at least one bounding box of the subset, the at least one bounding box being different from an outlier bounding box of the one or more outlier bounding boxes.

2 . The method of claim 1 , wherein the representing size of the bounding boxes is a median size of the bounding boxes.

3 . The method of claim 1 , wherein the predetermined second window comprises multiple images, wherein a total number of the multiple images in the predetermined second window is determined by a frame rate of an imaging device and a predetermined second of the predetermined second window.

4 . The method of claim 1 , wherein the plurality of images corresponds to a plurality of video frames of one or more videos.

5 . The method of claim 1 , wherein the N-frame buffer comprises the image and N preceding images of the image for the one or more bounding boxes.

6 . The method of claim 5 , wherein determining validity of each of the one or more bounding boxes in the image comprises:
determining whether a respective bounding box of the one or more bounding boxes between the image and an Nth image in the N-frame history buffer meets a first condition, determining whether the respective bounding box between the image and the Nth image in the N-frame history buffer meets a second condition; repeating to determine whether the respective bounding box between an (N−i)th image and (N−i−1)th image in the N-frame history buffer meet the first condition and the second condition, wherein the i is from 0 to N+2; and determining that the respective bounding box is valid when the respective bounding box meets the first condition and the second condition for each of the N−i images in the N-frame history buffer.

7 . The method of claim 6 , wherein the first condition for the respective bounding box between a first image and a second image is met when a current size of the respective bounding box in the first image changes less than a predetermined percentage of a previous size of the respective bounding box in the second image in the N-frame history buffer; and
wherein the second condition for the respective bounding box between the first image and the second image is met when a distance between a current centroid of the respective bounding box in the first image and a previous centroid of the respective bounding box in the second image in the N-frame history buffer is shorter than a predetermined distance.

8 . An insect trap and identification system comprising:
an imaging chamber; a memory; a processor with the memory configured to: receive a set of a plurality of images including one or more objects from the imaging chamber; generate one or more bounding boxes around a subset of the one or more objects in an image of the set of the plurality of images, the one or more bounding boxes corresponding to a subset one or more objects; generate an N-frame history buffer for each of the one or more bounding boxes in the image; determine validity of each of the one or more bounding boxes in the image based on the N-frame history buffer of a respective bounding box of the one or more bounding boxes; in response to the validity of each of the one or more bounding boxes in the image, select a subset of the one or more bounding boxes in the image; transmit one or more final bounding boxes based on the subset of the one or more bounding boxes in the image to a deep learning model; receive, from the deep learning model, one or more classifications of the one or more final bounding boxes; identifying a predetermined second window including the image; identifying a representing size of bounding boxes in the predetermined second window, the bounding boxes in the predetermined second window corresponding to each of the subset of the one or more bounding boxes in the image; identifying one or more outlier bounding boxes in the bounding boxes, each of the one or more outlier bounding boxes being a predetermined percentage larger or smaller than the representing size; and in response to a number of valid images in the predetermined second window being equal to or greater than a predetermined value, transmitting the bounding boxes in the predetermined second window to the deep learning model, wherein each of the valid images includes at least one bounding box of the subset, the at least one bounding box being different from an outlier bounding box of the one or more outlier bounding boxes.

9 . The insect trap and identification system of claim 8 , wherein the representing size of the bounding boxes is a median size of the bounding boxes.

10 . The insect trap and identification system of claim 8 , wherein the predetermined second window comprises multiple images, wherein a total number of the multiple images in the predetermined second window is determined by a frame rate of an imaging device and a predetermined second of the predetermined second window.

11 . The insect trap and identification system of claim 8 , wherein the plurality of images corresponds to a plurality of video frames of one or more videos.

12 . The insect trap and identification system of claim 8 , wherein the N-frame buffer comprises the image and N preceding images of the image for the one or more bounding boxes.

13 . The insect trap and identification system of claim 12 , wherein to determine validity of each of the one or more bounding boxes in the image, the processor is configured to:
determine whether a respective bounding box of the one or more bounding boxes between the image and an Nth image in the N-frame history buffer meets a first condition, determine whether the respective bounding box between the image and the Nth image in the N-frame history buffer meets a second condition; repeat to determine whether the respective bounding box between an (N−i)th image and (N−i−1)th image in the N-frame history buffer meet the first condition and the s

Assignees

• Penn State Res Found

Inventors

• Patch Harland
• Troncoso Aldas Nelson Daniel
• Homan Eric
• Narayanan Vijaykrishnan
• Mathis Codey
• Lee Chonghan
• Grozinger Christina

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V40/10

  Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title

• H04N7/18

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title