System and method for CNN layer sharing

Patent metadata

Field	Value
Publication number	US-10628683-B2
Application number	US-201715832548-A
Country	US
Kind code	B2
Filing date	Dec 5, 2017
Priority date	Dec 5, 2016
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

Methods, systems, and techniques for sharing layers between convolutional neural networks (CNNs). A data processing system may include a first and a second CNN that share a first group of layers. The first CNN may include the first group of layers in series with a second group of layers and be configured such that data for the first CNN is input to the first group of layers. The second CNN may include the first group of layers in series with a third group of layers and be configured such that data for the second CNN is also input to the first group of layers.

First claim

The invention claimed is:

1. A data processing system, comprising:
a first convolutional neural network (CNN) trained to perform a first task, wherein the first CNN comprises a first group of layers connected in series with a second group of layers and is configured such that data for the first CNN is input to the first group of layers; and
a second CNN trained to perform a second task, wherein the second CNN comprises the first group of layers connected in series with a third group of layers and is configured such that data for the second CNN is input to the first group of layers,

wherein the data for the first CNN comprises a first image and the data for the second CNN comprises a second image, and
wherein the first CNN is configured to perform a first task comprising generating a feature vector identifying a first type of object depicted in the first image, and the second CNN is configured to perform a second task comprising generating a feature vector identifying a second and different type of object depicted in the second image.

2. The system of claim 1 , wherein the first and the second CNNs are configured to receive the first and the second image as part of a first batch of image data and a second batch of image data, respectively, the first batch of image data comprising the first image and the second batch of image data comprising the second image.

3. The system of claim 2 , wherein each of the first and second batches of image data comprises a four dimensional data structure.

4. The system of claim 2 , wherein the first and second batches of image data are different.

5. The system of claim 2 , wherein the first and second batches of image data are the same batch of images.

6. The system of claim 5 , wherein the first and second CNNs are configured such that the first group of layers processes the first image and the second image, the second group of layers receives the first image after the first image has been processed by the first group of layers and not the second image after the second image has been processed by the first group of layers, and the third group of layers receives the second image after the second image has been processed by the first group of layers and not the first image after the first image has been processed by the first group of layers.

7. The system of claim 1 , further comprising a video capture device communicative with the first and second CNNs, wherein the video capture device is configured to generate the first and second images as portions of first and second video frames captured by the video capture device, respectively.

8. The system of claim 7 , wherein the video capture device is configured to process the first and second images using the first and second CNNs, respectively.

9. The system of claim 7 , further comprising a server that is communicative with the video capture device, wherein the video capture device is configured to send the first and second images to the server, and wherein the server is configured to process the first and second images using the first and second CNNs, respectively.

10. A data processing method, comprising:
processing a first batch of data using a first convolutional neural network (CNN), the first CNN comprising a first group of layers connected in series with a second group of layers, wherein the first batch of data is input to the first CNN via the first group of layers; and
processing a second batch of data using a second CNN, the second CNN comprising the first group of layers connected in series with a third group of layers, wherein the second batch of data is input to the second CNN via the first group of layers,

wherein the first batch of data comprises a first image and the second batch of data comprises a second image, and
wherein the first CNN performs a first task comprising generating a feature vector identifying a first type of object depicted in the first image, and the second CNN performs a second task comprising generating a feature vector identifying a second and different type of object depicted in the second image.

11. The method of claim 10 , wherein the first and the second CNNs are configured to receive the first and the second image as part of a first batch of image data and a second batch of image data, respectively, the first batch of image data comprising the first image and the second batch of image data comprising the second image.

12. The method of claim 11 , wherein each of the first and second batches of image data comprises a four dimensional data structure.

13. The method of claim 11 , wherein the first and second batches of image data are different.

14. The method of claim 11 , wherein the first and second batches of image data are the same batch of images.

15. The method of claim 14 , wherein the first group of layers processes the first image and the second image, the second group of layers receives the first image after the first image has been processed by the first group of layers and not the second image after the second image has been processed by the first group of layers, and the third group of layers receives the second image after the second image has been processed by the first group of layers and not the first image after the first image has been processed by the first group of layers.

16. The method of claim 10 , further comprising:
capturing, at a video capture device, first and second video frames;
generating, at the video capture device, the first and second images as portions of the first and second video frames, respectively; and
sending the first and second images to the first group of layers.

17. The method of claim 16 , wherein the first and second CNNs run on the video capture device.

18. The method of claim 16 , wherein the first and second images are sent from the video capture device to a server on which the first and second CNNs run.

19. A data processing system, comprising:
a processor; and
a memory device having stored thereon computer program code that is executable by the processor and that, when executed by the processor, causes the processor to perform a method comprising:
processing a first batch of data using a first convolutional neural network (CNN), the first CNN comprising a first group of layers connected in series with a second group of layers, wherein the first batch of data is input to the first CNN via the first group of layers; and
processing a second batch of data using a second CNN, the second CNN comprising the first group of layers connected in series with a third group of layers, wherein the second batch of data is input to the second CNN via the first group of layers,

wherein the first batch of data comprises a first image and the second batch of data comprises a second image, and
wherein the first CNN performs a first task comprising generating a feature vector identifying a first type of object depicted in the first image, and the second CNN performs a second task comprising generating a feature vector identifying a second and different type of object depicted in the second image.

20. The system of claim 19 , wherein the first and the second CNNs are configured to receive the first and the second image as part of a first batch of image data and a second batch of image data, respectively, the first batch of image data comprising the first image and the second batch of image data comprising the second image.

21. The system of claim 20 , wherein each of the first and second batches of imag

Assignees

• Avigilon Corp

Inventors

• Doumbouya Moussa
• He Lu
• Saptharishi Mahesh

Classifications

• G06N3/08

  Learning methods · CPC title

• G06N3/084 Primary

  Backpropagation, e.g. using gradient descent · CPC title

• G06K9/00979

  Physics · mapped topic

• G06K9/6262

  Physics · mapped topic

• G06K9/00771 Primary

  Physics · mapped topic