Statistically-driven Coral Reef metaheuristic for automatic hyperparameter setting and architecture design of Convolutional Neural Networks

Authors

Alejandro Martín

Raúl Lara-Cabrera

Víctor Manuel Vargas-Yun

Pedro Antonio Gutiérrez

César Hervás-Martínez

David Camacho

Conference Proceedings

Proceedings of the 2020 IEEE congress on evolutionary computation (CEC2020)

ISBN 978-1-7281-6926-2

Pages 1–8

Abstract

The adjustment of the hyperparameters and network structure of Convolutional Neural Networks (CNNs) composes an important step towards building effective, but still efficient learning models. The selection of the best configuration is a problem-dependent task that involves to explore an enormous and complex search space. Due to this reason, the use of heuristicbased search fits perfectly within this task, seeking to obtain a near to optimal solution in a complex and large exploratory space. This paper presents SCRODeep, a self-adapting algorithm based on a statistically-driven Coral Reef Optimisation algorithm (SCRO), for the selection of the most adequate CNNs architecture in a particular domain. This metaheuristic has been designed to navigate through a search space where the architecture (defining the particular set of layers, including convolutional or pooling layers), and the hyperparameters of the network (i.e. activation functions, number of units or the kernel initializer, among others) are represented, but where the connections weights and bias are inferred using typical CNNs optimisation algorithms. In contrast to other approaches, where the use of a metaheuristic implies in turn to fix a series of hyperparameters (i.e. the mutation probability in a genetic algorithm), our approach follows a selfparametrisation perspective, thus removing the necessity of fixing these values. The method has been tested in the design of CNNs for image classification, showing that SCRODeep is able to find competitive solutions, while the complexity of the architectures found is constrained.

Keywords

architecture definition convolutional neural networks coral reef based optimisation optimisation

BibTex Citation

@inproceedings{Martin2020Statistically,
	author = {Mart{\' i}n, Alejandro and Lara-Cabrera, Ra{\' u}l and Vargas-Yun, V{\' i}ctor Manuel and Guti{\' e}rrez, Pedro Antonio and Herv{\' a}s-Mart{\' i}nez, C{\' e}sar and Camacho, David},
	booktitle = {Proceedings of the 2020 {IEEE} congress on evolutionary computation ({CEC2020})},
	doi = {10.1109/CEC48606.2020.9185914},
	year = {2020},
	pages = {1--8},
	title = {Statistically-driven {Coral} {Reef} metaheuristic for automatic hyperparameter setting and architecture design of {Convolutional} {Neural} {Networks}},
	url = {https://doi.org/10.1109/CEC48606.2020.9185914},
	howpublished = {https://doi.org/10.1109/CEC48606.2020.9185914},
}

BibTex Unicode Citation

@inproceedings{Martin2020Statistically,
	author = {Martín, Alejandro and Lara-Cabrera, Raúl and Vargas-Yun, Víctor Manuel and Gutiérrez, Pedro Antonio and Hervás-Martínez, César and Camacho, David},
	booktitle = {Proceedings of the 2020 {IEEE} congress on evolutionary computation ({CEC2020})},
	doi = {10.1109/CEC48606.2020.9185914},
	year = {2020},
	pages = {1--8},
	title = {Statistically-driven {Coral} {Reef} metaheuristic for automatic hyperparameter setting and architecture design of {Convolutional} {Neural} {Networks}},
	url = {https://doi.org/10.1109/CEC48606.2020.9185914},
	howpublished = {https://doi.org/10.1109/CEC48606.2020.9185914},
}

APA Citation

Martín, A., Lara-Cabrera, R., Vargas-Yun, V. M., Gutiérrez, P. A., Hervás-Martínez, C., & Camacho, D. (2020). Statistically-driven Coral Reef metaheuristic for automatic hyperparameter setting and architecture design of Convolutional Neural Networks. Proceedings of the 2020 IEEE Congress on Evolutionary Computation (CEC2020), 1–8. https://doi.org/10.1109/CEC48606.2020.9185914

RIS Citation

TY  - CONF
AU  - Martín, Alejandro
AU  - Lara-Cabrera, Raúl
AU  - Vargas-Yun, Víctor Manuel
AU  - Gutiérrez, Pedro Antonio
AU  - Hervás-Martínez, César
AU  - Camacho, David
C3  - Proceedings of the 2020 IEEE congress on evolutionary computation (CEC
2020)
DA  - 2020///
C2  - 2020
DO  - 10.1109/CEC48606.2020.9185914
ID  - temp_id_170803284618
SP  - 1-8
TI  - Statistically-driven Coral Reef metaheuristic for automatic hyperparam
eter setting and architecture design of Convolutional Neural Networks
UR  - https://doi.org/10.1109/CEC48606.2020.9185914
ER  -

CV Citation

A. Martín, R. Lara-Cabrera, V.M. Vargas-Yun, P.A. Gutiérrez, C. Hervás-Martínez, D. Camacho (3/6). "Statistically-driven Coral Reef metaheuristic for automatic hyperparameter setting and architecture design of Convolutional Neural Networks". Proceedings of the 2020 IEEE congress on evolutionary computation (CEC2020), pp. 1–8, 2020.