Statistically-driven coral reef metaheuristic for automatic hyperparameter setting and architecture design of convolutional neural networks

A. Martín , R. Lara-Cabrera, V. Vargas, P. Gutiérrez, C. Hervás-Martínez, D. Camacho

2020 IEEE congress on evolutionary computation (CEC), pp. 1-8, 2020

Abstract

The adjustment of the hyperparameters and network structure of Convolutional Neural Networks (CNNs) composes an important step towards building effective, but still efficient learning models. The selection of the best configuration is a problem-dependent task that involves to explore an enormous and complex search space. Due to this reason, the use of heuristic-based search fits perfectly within this task, seeking to obtain a near to optimal solution in a complex and large exploratory space. This paper presents SCRODeep, a self-adapting algorithm based on a statistically-driven Coral Reef Optimisation algorithm (SCRO), for the selection of the most adequate CNNs architecture in a particular domain. This metaheuristic has been designed to navigate through a search space where the architecture (defining the particular set of layers, including convolutional or pooling layers), and the hyperparameters of the network (i.e. activation functions, number of units or the kernel initializer, among others) are represented, but where the connections weights and bias are inferred using typical CNNs optimisation algorithms. In contrast to other approaches, where the use of a metaheuristic implies in turn to fix a series of hyperparameters (i.e. the mutation probability in a genetic algorithm), our approach follows a self-parametrisation perspective, thus removing the necessity of fixing these values. The method has been tested in the design of CNNs for image classification, showing that SCRODeep is able to find competitive solutions, while the complexity of the architectures found is constrained.

Cite this publication

BibTex

@inproceedings{martin2020statistically-driven,
    author = {Alejandro Martín and Raúl Lara-Cabrera and Víctor Manuel Vargas and Pedro Antonio Gutiérrez and César Hervás-Martínez and David Camacho},
    title = {Statistically-driven coral reef metaheuristic for automatic hyperparameter setting and architecture design of convolutional neural networks},
    booktitle = {2020 IEEE congress on evolutionary computation (CEC)},
    year = {2020},
    pages = {1--8},
    doi = {10.1109/CEC48606.2020.9185914}
}

APA

Martín, A., Lara-Cabrera, R., Vargas, V., Gutiérrez, P., Hervás-Martínez, C., Camacho, D. (2020). Statistically-driven coral reef metaheuristic for automatic hyperparameter setting and architecture design of convolutional neural networks. In 2020 IEEE congress on evolutionary computation (CEC) (pp. 1-8).

A. Martín (CA), R. Lara-Cabrera, V.M. Vargas, P.A. Gutiérrez, C. Hervás-Martínez, D. Camacho, (3/6) "Statistically-driven coral reef metaheuristic for automatic hyperparameter setting and architecture design of convolutional neural networks". 2020 IEEE congress on evolutionary computation (CEC), pp. 1-8, 2020.

RIS

TY  - CONF
T1  - Statistically-driven coral reef metaheuristic for automatic hyperparameter setting and architecture design of convolutional neural networks
T2  - 2020 IEEE congress on evolutionary computation (CEC)
AU  - Martín, Alejandro
AU  - Lara-Cabrera, Raúl
AU  - Vargas, Víctor Manuel
AU  - Gutiérrez, Pedro Antonio
AU  - Hervás-Martínez, César
AU  - Camacho, David
JO  - 2020 IEEE congress on evolutionary computation (CEC)
JA  - 2020 IEEE congress on evolutionary computation (CEC)
Y1  - 2020
PY  - 2020
SP  - 1
EP  - 8
DO  - 10.1109/CEC48606.2020.9185914
ER  -