Multi-task based lifelong learning

Patent metadata

Field	Value
Publication number	US-2020175362-A1
Application number	US-201916379704-A
Country	US
Kind code	A1
Filing date	Apr 9, 2019
Priority date	Nov 30, 2018
Publication date	Jun 4, 2020
Grant date	—

Abstract

Methods, devices, and computer-readable media for multi-task based lifelong learning. A method for lifelong learning includes identifying a new task for a machine learning model to perform. The machine learning model trained to perform an existing task. The method includes adaptively training a network architecture of the machine learning model to generate an adapted machine learning model based on incorporating inherent correlations between the new task and the existing task. The method further includes using the adapted machine learning model to perform both the existing task and the new task.

First claim

What is claimed is:

1 . A method for lifelong learning, the method comprising:
identifying a new task for a machine learning model to perform, the machine learning model trained to perform an existing task; adaptively training a network architecture of the machine learning model to generate an adapted machine learning model based on incorporating inherent correlations between the new task and the existing task; and using the adapted machine learning model to perform both the existing task and the new task.

2 . The method of claim 1 , further comprising:
expanding a size of the network architecture of the machine learning model using AutoML.

3 . The method of claim 2 , wherein expanding the size of the network architecture of the machine learning model using AutoML comprises using wider and deeper operators by:
adding a layer to the network architecture; and expanding one or more existing layers of the network architecture.

4 . The method of claim 3 , further comprising:
identifying the added layer as a task-specific layer for the new task.

5 . The method of claim 2 , further comprising:
compressing the network architecture of the machine learning model to reduce the size.

6 . The method of claim 1 , wherein the machine learning model is a compressed model.

7 . The method of claim 2 , further comprising:
training the machine learning model to perform the new task using training data for the new task; and compressing the expanded network architecture of the trained machine learning model using the training data for the new task.

8 . An electronic device for lifelong learning, the electronic device comprising:
a memory configured to store a machine learning model trained to perform an existing task; and a processor operably connected to the memory, the processor configured to:
identify a new task for the machine learning model to perform;
adaptively train a network architecture of the machine learning model to generate an adapted machine learning model based on incorporating inherent correlations between the new task and the existing task; and
use the adapted machine learning model to perform both the existing task and the new task.

9 . The electronic device of claim 8 , wherein the processor is further configured to:
expand a size of the network architecture of the machine learning model using AutoML.

10 . The electronic device of claim 9 , wherein to expand the size of the network architecture of the machine learning model using AutoML, the processor is further configured to use wider and deeper operators to:
add a layer to the network architecture; and expand one or more existing layers of the network architecture.

11 . The electronic device of claim 10 , wherein the processor is further configured to:
identify the added layer as a task-specific layer for the new task.

12 . The electronic device of claim 9 , wherein the processor is further configured to:
compress the network architecture of the machine learning model to reduce the size.

13 . The electronic device of claim 8 , wherein the machine learning model is a compressed model.

14 . The electronic device of claim 9 , wherein the processor is further configured to:
train the machine learning model to perform the new task using training data for the new task; and compress the expanded network architecture of the trained machine learning model using the training data for the new task.

15 . A non-transitory, computer-readable medium comprising program code for lifelong learning that, when executed by a processor of an electronic device, causes the electronic device to:
identify a new task for a machine learning model to perform, the machine learning model trained to perform an existing task; adaptively train a network architecture of the machine learning model to generate an adapted machine learning model based on incorporating inherent correlations between the new task and the existing task; and use the adapted machine learning model to perform both the existing task and the new task.

16 . The non-transitory, computer-readable medium of claim 15 , further comprising program code that, when executed by the processor, causes the electronic device to:
expand a size of the network architecture of the machine learning model using AutoML.

17 . The non-transitory, computer-readable medium of claim 16 , wherein the program code that, when executed, causes the electronic device to expand the size of the network architecture of the machine learning model using AutoML comprises program code that, when executed by the processor, causes the electronic device to use wider and deeper operators to:
add a layer to the network architecture; and expand one or more existing layers of the network architecture.

18 . The non-transitory, computer-readable medium of claim 17 , further comprising program code that, when executed by the processor, causes the electronic device to:
identify the added layer as a task-specific layer for the new task.

19 . The non-transitory, computer-readable medium of claim 16 , further comprising program code that, when executed by the processor, causes the electronic device to:
compress the network architecture of the machine learning model to reduce the size.

20 . The non-transitory, computer-readable medium of claim 16 , further comprising program code that, when executed by the processor, causes the electronic device to:
train the machine learning model to perform the new task using training data for the new task; and compress the expanded network architecture of the trained machine learning model using the training data for the new task.

Assignees

• Samsung Electronics Co Ltd

Inventors

• Zhang Jie
• Zhang Junting
• Ghosh Shalini
• Li Dawei
• Zhu Jingwen

Classifications

• G06N3/08 Primary

  Learning methods · CPC title

• G06N3/04

  Architecture, e.g. interconnection topology · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N3/082 Primary

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/096

  Transfer learning · CPC title