Multiobjective coevolution of deep neural network architectures

Patent metadata

Field	Value
Publication number	US-2025232173-A1
Application number	US-202519171428-A
Country	US
Kind code	A1
Filing date	Apr 7, 2025
Priority date	Nov 1, 2018
Publication date	Jul 17, 2025
Grant date	—

Abstract

An evolutionary AutoML framework called LEAF optimizes hyperparameters, network architectures and the size of the network. LEAF makes use of both evolutionary algorithms (EAs) and distributed computing frameworks. A multiobjective evolutionary algorithm is used to maximize the performance and minimize the complexity of the evolved networks simultaneously by calculating the Pareto front given a group of individuals that have been evaluated for multiple objectives.

First claim

We claim:

1 . A method for co-evolution of deep neural network (DNN) hyperparameters and architecture in accordance with multiple objective optimization, the method comprising:
evaluating assembled DNNs in an initial population in accordance with a primary and secondary objective, wherein each assembled DNN in the initial population is comprised of modules and is assembled in accordance with an architecture blueprint, and further wherein the evaluation results in selection of a first group of selected assembled DNNs and a second group of leftover assembled DNNs; for each subsequent generation of an evolution run:
creating an archive of assembled DNNs from the second group of leftover assembled DNNs from a last generation;
evaluating each assembled DNN in the archive based on each of the primary and secondary objectives;
determining a Pareto front of assembled DNN in the archive;
removing assembled DNNs in the Pareto front and adding the removed DNNs to a new archive; and
truncating the new archive by removing a last fraction of added assembled DNNs;
wherein the method for co-evolution ends when values for at least the primary objective converge.

2 . The method of claim 1 , wherein determining Pareto front of the assembled DNN in the archive includes:
sorting the assembled DNN in the archive in a descending order by the primary objective; creating a new Pareto front with a first individual assembled DNN in the archive; and for each successive individual,
if a corresponding value for a secondary objective of the successive individual is greater than that of a preceding individual, adding the successive individual to the Pareto front.

3 . The method of claim 2 , further including sorting the Pareto front in a descending order based on values of the secondary objective.

4 . The method of claim 1 , wherein the primary objective is fitness and the secondary objective is novelty.

5 . The method of claim 1 , wherein the novelty is a behavior vector of multiple concatenated features.

6 . The method of claim 1 , wherein the concatenated features include:
number of parameters of the assembled DNN; number of layers (L) in the assembled DNN; number of connections between layers (C); length of a longest path within the assembled DNN; density of the assembled DNN, defined as D=|C|/|L|(|L|−1) a maximum of the number of connections for each layer; a mean of the number of connections for each layer; a standard deviation of the number of connections for each layer; a maximum of page rank for each layer; a mean of page rank for each layer; and standard deviation of page rank for each layer.

Assignees

• Cognizant Tech Solutions U S Corporation

Inventors

• Liang Jason Zhi
• Meyerson Elliot
• Miikkulainen Risto

Classifications

• G06N3/0985

  Hyperparameter optimisation; Meta-learning; Learning-to-learn · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/082

  modifying the architecture, e.g. adding, deleting or silencing nodes or connections · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title