Groupe d’études et de recherche en analyse des décisions

# DGR-ELM -- Distributed generalized regularized ELM for classification

## Fernando Kentaro Inaba, Evandro Ottoni Teatini Salles, Sylvain Perron et Gilles Caporossi

Extreme Learning Machine (ELM) has recently increased popularity and has been successfully applied to a wide range of applications. Variants using regularization are now a common practice in the state of the art in ELM field. The most commonly used regularization is the $$\ell_2$$ norm which improves generalization but result in a dense network. Regularization based on the elastic net has also been proposed but mainly applied to regression and binary classification problems. In this paper, we propose a generalization of regularized ELM (R-ELM) for multiclass classification problems, termed GR-ELM. We achieve such generalization using the $$\ell_2,_1$$ and Frobenius norm. Traditional R-ELM is a particular case of our method when binary classification tasks are considered. We also propose an alternative algorithm for GR-ELM when training data is distributed, namely GR-ELM. We use alternating direction method of multipliers (ADMM) for solving the resulting optimization problems. Message passing interface (MPI) in a single program, multiple data (SPMD) programming style is chosen for implementing DGR-ELM. Extensive experiments are conducted to evaluate the proposed method. Our experiments show that GR-ELM and DGR-ELM have similar training and testing accuracy when compared to R-ELM, although usually faster testing time is obtained with our method due to the compactness of the resulting network.

, 19 pages