Recycling system and method based on deep-learning and computer vision technology

Patent metadata

Field	Value
Publication number	US-10824936-B2
Application number	US-201916259245-A
Country	US
Kind code	B2
Filing date	Jan 28, 2019
Priority date	Aug 13, 2018
Publication date	Nov 3, 2020
Grant date	Nov 3, 2020

Abstract

A recycling system and a method based on deep-learning and computer vision technology are disclosed. The system includes a trash sorting device and a trash sorting algorithm. The trash sorting device includes a trash arraying mechanism, trash sensors, a trash transfer mechanism and a controller. The trash arraying mechanism is configured to process trash in a batch manner. The controller drives the trash arraying mechanism according to the signals of trash sensors and controls the sorting gates of the trash sorting mechanism to rotate. The trash sorting algorithm makes use of the images of trash, wherein the images are taken by cameras in different directions. The trash sorting algorithm includes a dynamic object detection algorithm, an image pre-processing algorithm, an identification module and a voting and selecting algorithm. The identification module is based on the convolutional neural networks (CNNs) and may at least identify four kinds of trash.

First claim

What is claimed is:

1. A recycling system based on deep learning and computer vision technology, comprising:
a plurality of cameras, respectively configured to photograph a trash object in a plurality of directions and producing a plurality of trash object images;
a controller, connected to the plurality of cameras to receive the plurality of trash object images for identifying a type of the trash object, the controller comprising a data storage storing computer codes including:
a plurality of artificial neural network models, each trained by images taken in a direction corresponding to the plurality of directions, and identifying the trash object according to the plurality of trash object images to generate a judgment of the type of the trash object; and
a voting and selecting algorithm, configured to combine the judgment of each artificial neural network model for generating a determination of the type of the trash object; and

a trash sorting device, configured to receive the trash object and connected to and controlled by the controller;
wherein the trash sorting device delivers the trash object to a trash channel corresponding to the type of the trash object determined by the voting and selecting algorithm.

2. The recycling system according to claim 1 , wherein the computer codes further include a dynamic object detection algorithm, the dynamic object detection algorithm processes an image subtraction of the plurality of trash object images taken at a time point and a next time point after the time point for detecting whether or not the trash obj ect is passing by the plurality of cameras; if the trash object is passing by, the dynamic object detection algorithm generates a command to drive the plurality of artificial neural network models and the voting and selecting algorithm for identifying the type of the trash object passing by the plurality of cameras; if the trash object is not passing by, the plurality of artificial neural network models and the voting and selecting algorithm are not driven.

3. The recycling system according to claim 2 , wherein, the trash sorting device further comprises a metal sensor connected to the controller; the metal sensor detects whether or not the trash object is metallic and outputs a metal detecting signal to the controller; and the controller identifies the type of the trash object according to the metal detecting signal and the determination of the voting and selecting algorithm.

4. The recycling system according to claim 3 , wherein the trash sorting device further comprising:
a cover, disposed over the trash sorting device to block external light; and
a light source, disposed under the cover to provide stable light and improve an identification accuracy of the trash object.

5. The recycling system according to claim 4 , wherein the trash sorting device further comprising:
a trash detector, connected to the controller and sensing whether or not the trash object enters the trash sorting device, wherein if the trash object enters, the trash detector transmits a detecting signal to the controller;
a stirrer, connected to the controller, wherein when receiving the detecting signal, the controller controls the stirrer to stir the trash object for reducing an overlap situation where the trash object overlaps with adjacent objects, that achieves a function of batch processing the trash object and further improves the identification accuracy of the trash object; and
a control gate, connected to the controller; wherein when receiving the detecting signal, the controller controls the control gate to identify the trash object in a batch manner.

6. A computer implemented recycling method based on deep learning and computer vision technology for a trash sorting device, the recycling method comprising:
configuring a plurality of artificial neural network models in a controller, wherein each artificial neural network model is trained by images taken in a direction corresponding to a plurality of directions toward which a plurality of cameras are respectively configured;
configuring the cameras to photograph a trash object in the plurality of directions respectively and generate a plurality of trash object images;
connecting the plurality of cameras to the controller, and transmitting the plurality of trash object images to the controller;
identifying a type of the trash object as an identification result according to the plurality of trash object images through each artificial neural network model;
combining the identification results of the plurality of artificial neural network models and transmitting the identification results to a voting and selecting algorithm configured in the controller and connected to the plurality of artificial neural network models for generating a determination of the type of the trash object; and
delivering the trash object to a trash channel corresponding to the determination of the voting and selecting algorithm through a trash sorting device connected to and controlled by the controller.

7. The recycling method according to claim 6 , further comprising:
configuring a dynamic object detection algorithm in the controller and processing an image subtraction of the plurality of trash object images taken at a time point and a next time point after the time point for detecting whether or not the trash object is passing by the plurality of cameras through the dynamic object detection algorithm; if the trash object is passing by, the dynamic object detection algorithm generates a command to drive the plurality of artificial neural network models and the voting and selecting algorithm for identifying the type of the trash object passing by the plurality of cameras; if the trash object is not passing by, the plurality of artificial neural network models and the voting and selecting algorithm are not driven.

8. The recycling method according to claim 7 , further comprising:
connecting a metal sensor to the controller, using the metal sensor to sense whether or not the trash object is metallic, transmitting a metal detecting signal from the metal sensor to the controller, and
combining the metal detecting signal and the determination of the voting and selecting algorithm to identify the type of the trash object.

9. The recycling method according to claim 8 , further comprising:
disposing a cover over the trash sorting device to block external light; and
disposing a light source under the cover to provide stable light for improving an identification accuracy of the trash object.

10. The recycling method according to claim 9 , further comprising:
configuring a trash detector, a stirrer and a control gate on the trash sorting device, and respectively connecting the trash detector, the stirrer and the control gate to the controller;
sensing whether or not the trash object enters the trash sorting device using the trash detector, wherein if the trash object enters, the trash detector transmits a detecting signal to the controller; and
controlling the control gate to identify the trash object in a batch manner and controlling the stirrer to stir the trash object for reducing an overlap situation where the trash object overlaps with adjacent objects through the controller for improving the identification accuracy of the trash object.

Assignees

• Univ National Chiao Tung

Inventors

• Wu Bing-Fei
• Tseng Wan-Ju
• Chen Yu-Ming
• Wu Bing-Jhang
• Wu Yi-Chiao
• Chung Meng-Liang

Classifications

• B07C5/36 Primary

  Sorting apparatus characterised by the means used for distribution {(sorting according to destination B07C3/003, B07C3/02)} · CPC title

• G06V10/82

  using neural networks · CPC title

• G06V10/809

  of classification results, e.g. where the classifiers operate on the same input data · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title