System and method for continual learning using experience replay

Patent metadata

Field	Value
Publication number	US-11645544-B2
Application number	US-202016875852-A
Country	US
Kind code	B2
Filing date	May 15, 2020
Priority date	Jul 17, 2019
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

Described is a system for continual learning using experience replay. In operation, the system receives a plurality of tasks sequentially, from which a current task is fed to an encoder. The current task has data points associated with the current task. The encoder then maps the data points into an embedding space, which reflects the data points as discriminative features. A decoder then generates pseudo-data points from the discriminative features, which are provided back to the encoder. The discriminative features are updated in the embedding space based on the pseudo-data points. The encoder then learns (updates) a classification of a new task by matching the new task with the discriminative features in the embedding space.

First claim

What is claimed is:

1. A system for continual learning using experience replay, the system comprising:
one or more processors and a memory, the memory being a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions, the one or more processors perform operations of:
receiving a plurality of tasks sequentially, the plurality of tasks having at least a current task with associated data points;
mapping the data points into an embedding space, the embedding space reflecting the data points as discriminative features;
generating pseudo-data points from the discriminative features;
updating the discriminative features in the embedding space based on the pseudo-data points; and
updating a classification of a new task by matching the new task with the discriminative features in the embedding space.

2. The system as set forth in claim 1 , wherein updating the classification of the new task is performed using a stochastic gradient descent process.

3. The system as set forth in claim 2 , wherein in updating the discriminative features in the embedding space, a sliced-Wasserstein distance is used to enforce all tasks to share a same distribution in the embedding space.

4. The system as set forth in claim 3 , further comprising operations of:
identifying an object in a field-of-view of an autonomous vehicle based on the classification of the new task; and
causing the autonomous vehicle to perform an operation based on the object identification.

5. The system as set forth in claim 1 , wherein in updating the discriminative features in the embedding space, a sliced-Wasserstein distance is used to enforce all tasks to share a same distribution in the embedding space.

6. The system as set forth in claim 1 , further comprising operations of:
identifying an object in a field-of-view of an autonomous vehicle based on the classification of the new task; and
causing the autonomous vehicle to perform an operation based on the object identification.

7. A computer program product for continual learning using experience replay, the computer program product comprising:
a non-transitory computer-readable medium having executable instructions encoded thereon, such that upon execution of the instructions by one or more processors, the one or more processors perform operations of:
receiving a plurality of tasks sequentially, the plurality of tasks having at least a current task with associated data points;
mapping the data points into an embedding space, the embedding space reflecting the data points as discriminative features;
generating pseudo-data points from the discriminative features;
updating the discriminative features in the embedding space based on the pseudo-data points; and
updating a classification of a new task by matching the new task with the discriminative features in the embedding space.

8. The computer program product as set forth in claim 7 , wherein updating the classification of the new task is performed using a stochastic gradient descent process.

9. The computer program product as set forth in claim 8 , wherein in updating the discriminative features in the embedding space, a sliced-Wasserstein distance is used to enforce all tasks to share a same distribution in the embedding space.

10. The computer program product as set forth in claim 9 , further comprising operations of:
identifying an object in a field-of-view of an autonomous vehicle based on the classification of the new task; and
causing the autonomous vehicle to perform an operation based on the object identification.

11. The computer program product as set forth in claim 7 , wherein in updating the discriminative features in the embedding space, a sliced-Wasserstein distance is used to enforce all tasks to share a same distribution in the embedding space.

12. The computer program product as set forth in claim 7 , further comprising operations of:
identifying an object in a field-of-view of an autonomous vehicle based on the classification of the new task; and
causing the autonomous vehicle to perform an operation based on the object identification.

13. A computer implemented method for continual learning using experience replay, the method comprising an act of:
causing one or more processers to execute instructions encoded on a non-transitory computer-readable medium, such that upon execution, the one or more processors perform operations of:
receiving a plurality of tasks sequentially, the plurality of tasks having at least a current task with associated data points;
mapping the data points into an embedding space, the embedding space reflecting the data points as discriminative features;
generating pseudo-data points from the discriminative features;
updating the discriminative features in the embedding space based on the pseudo-data points; and
updating a classification of a new task by matching the new task with the discriminative features in the embedding space.

14. The method as set forth in claim 13 , wherein updating the classification of the new task is performed using a stochastic gradient descent process.

15. The method as set forth in claim 14 , wherein in updating the discriminative features in the embedding space, a sliced-Wasserstein distance is used to enforce all tasks to share a same distribution in the embedding space.

16. The method as set forth in claim 15 , further comprising acts of:
identifying an object in a field-of-view of an autonomous vehicle based on the classification of the new task; and
causing the autonomous vehicle to perform an operation based on the object identification.

17. The method as set forth in claim 13 , wherein in updating the discriminative features in the embedding space, a sliced-Wasserstein distance is used to enforce all tasks to share a same distribution in the embedding space.

18. The method as set forth in claim 13 , further comprising acts of:
identifying an object in a field-of-view of an autonomous vehicle based on the classification of the new task; and
causing the autonomous vehicle to perform an operation based on the object identification.

Assignees

• Hrl Lab Llc

Inventors

• Rostami Mohammad
• Kolouri Soheil
• Pilly Praveen K

Classifications

• G06N3/0475

  Generative networks · CPC title

• G06N3/0455

  Auto-encoder networks; Encoder-decoder networks · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/045

  Combinations of networks · CPC title

• G06N7/01

  Probabilistic graphical models, e.g. probabilistic networks · CPC title