Randomized Neural Networks, Pseudo-Inverse Based Solutions and Non-iterative Learning

This page presents our research centered around randomized neural networks, Moore-Penrose pseudo-inverse (MPPI) based closed-form solutions for training neural networks and related topics.

1.    L. Zhang, P. N. Suganthan, "A Comprehensive Evaluation of Random Vector Functional Link Networks," Information Sciences, DOI: 10.1016/j.ins.2015.09.025.  Software Available here.

In the context of classification, the above work shows that the direct links in RVFL improve the performance of the single layer feedforward Network (SLFN) when randomized weights are used between inputs and hidden neurons, suggesting that direct-links regularize the effects of randomization. In other words, instead of using only randomized features, mixing the randomized features with the original features performs better. Further, randomization range can also be tuned to improve performance.121 UCI datasets were used in our study. Our experimental framework is identical to the one used in the following publication:
M. Fernández-Delgado, E. Cernadas, S. Barro, D. Amorim, "Do we Need Hundreds of Classifiers to Solve Real World Classification Problems?", JMLR, 15(Oct):3133−3181, 2014.

2.     Y. Ren, P.  N. Suganthan, N. Srikanth, G. Amaratunga, "Random Vector Functional Link Network for Short-term Electricity Load Demand Forecasting", Information Sciences, Software Available here.   Also from: https://github.com/ron1818/PhD_code/
In the context of time series forecasting, the above work shows that the direct links in RVFL improve the performance in a statistically significant manner. The direct links can be compared to the time delayed line in the finite impulse response filter (FIR). Randomization range can also be tuned to improve performance. RVFL with direct input-output links slightly outperformed SVR also. 18 time series datasets were used in this study.


3.      L. Zhang, P. N. Suganthan, "A Survey of Randomized Algorithms for Training Neural Networks," Information Sciences, DoI: 10.1016/j.ins.2016.01.039, accepted, 2016.


Promising Future Research Direction: There is an increase in research activities using randomized neural networks. Researchers are encouraged to further investigate the role of direct links in RVFL and other randomized neural network configurations.