Mr
Martin Beseda
(IT4Innovations, VSB-TUO)Mr
Martin Mrovec
(IT4Innovations, VSB-TUO)
Machine Learning is a rapidly developing scientific discipline with applications in many other fields. We are interested in the applications of Artificial Neural Networks (ANNs) [1] in physical chemistry and, subsequently, in plasma medicine. Within the scope of the computational chemistry, there are several privileged applications, namely potential energy surfaces representations, solving of differential equations and molecular clusters categorization.
For such purposes, it is usually necessary to use very large ANNs which tend to be very computationally demanding, up to the point where they cannot be trained or evaluated on a personal computer in a reasonable time. Also, some applications require a specific architecture of the neural network to work correctly. Unfortunately, those problems are not covered by state-of-the-art software, which either runs only sequentially and thus it is not computationally-efficient enough or it offers only specialized applications, i.e. a very limited network architecture.
That led us to the idea of developing our own massively-parallel ANN library, which will allow users to specify their own, arbitrary, architecture, so it might be used in a wide range of applications, overcoming time and memory limitations by utilizing large modern infrastructures to the fullest.
Our library *lib4neuro* is implemented in a way that enables us to easily implement and to try new training algorithms, which seem to be one of the most significant aspects considering time-consumption. Today, not only usual back-propagation is being used, but also two global optimization methods are implemented now - particle swarm [2] and simulated annealing [3]. All these methods have a great potential to be efficiently parallelized. lib4neuro is going to utilize not only the common MPI+OpenMP combination but also OpenACC to be able to use GPUs and MICs to off-load computationally-heavy parts. Moreover, MPI will be partly substituted by DASH, an innovative way of distributive-level parallelization developed in HLRS institute [4].
With network flexibility being our priority, lib4neuro is able to construct, train and evaluate very atypical network architectures to the point where an arbitrary number of networks with mutually dependent weights can be constructed and then trained simultaneously. This allows us, for example, to formulate differential equations with initial and boundary conditions using neural networks only and solve them efficiently. lib4neuro is also able to construct ANNs with fully arbitrary connections where no interconnection graph completeness is needed neither for training nor evaluation. With such an architecture, the network can be trained much more efficiently.
Also, modified versions of network-processing, inspired by Hopfield networks, Kohonen maps, and convolutional networks are being implemented so it will be possible to use lib4neuro for image-classification or detection of patterns in data. Those features will be further optimized to be used efficiently for the above-mentioned molecular cluster categorization.
So far, easy-to-use interface for solving differential equations has been fully implemented and its results will be presented.
Summary
Machine Learning is a rapidly developing scientific discipline with applications in many other fields. We are interested in the applications of Artificial Neural Networks (ANNs) in physical chemistry and, subsequently, in plasma medicine.
For such purposes, it is usually necessary to use very large ANNs which tend to be very computationally demanding. Also, some applications require a specific architecture of the neural network to work correctly. Unfortunately, those problems are not covered by state-of-the-art software, which is either not fast enough or it offers only a few possible ANN architectures.
That led us to the idea of developing our own massively-parallel ANN library, which will allow users to specify their own, arbitrary, architecture, so it might be used in a wide range of applications, overcoming time and memory limitations by utilizing large modern infrastructures to the fullest.
References
[1] Schalkoff, Robert J. Artificial neural networks. Vol. 1. New York: McGraw-Hill, 1997.
[2] Marini, Federico, and Beata Walczak. "Particle swarm optimization (PSO). A tutorial." Chemometrics and Intelligent Laboratory Systems 149 (2015): 153-165.
[3] Kirkpatrick, Scott, C. Daniel Gelatt, and Mario P. Vecchi. "Optimization by simulated annealing." science 220.4598 (1983): 671-680.
[4] www.hlrs.de
Mr
Martin Beseda
(IT4Innovations, VSB-TUO)
Mr
Martin Mrovec
(IT4Innovations, VSB-TUO)
Mr
Michal Kravčenko
(IT4Innovations, VSB-TUO)
