This two day course covers topics that arise in the use of R on large data for data science computations on HPC systems. Aimed at intermediate level for current R users who want to speed up and scale up their codes on larger platforms. The course begins with some basics but quickly moves on to strategies to make existing R code faster and toward the use of HPC parallel platforms. There will be code demonstrations and hands-on exercises.
Purpose of the course (benefits for the attendees)
The course aims to demonstrate and teach how R can be used for truly large data on large systems. While R is a high-level scripting language where small missteps can be costly in execution time and memory use, more careful coding practices can lead to more efficient use of what is often compiled code underneath. The course will cover intermediate to advanced aspects of R and relevant packages for data science aimed at the utilization of high performance computing resources. Participants will learn strategies to make R code faster, methods to include new compiled code, methods to utilize multiple cores for parallel speedup, and methods to distribute data and compute on large distributed platforms with the pbdR package ecosystem.
Participants familiar with HPC resources and without a substantial R background can also benefit. Advance preparation with some introductory resources on R, which are widely available on the web as well as in many books, will be helpful. Consider e.g.
- R programming for those coming from other languages: http://www.johndcook.com/R_language_for_programmers.html
- aRrgh - a newcomer's (angry) guide to R: http://tim-smith.us/arrgh/
- Book The Art of R Programming by Norm Matloff: http://nostarch.com/artofr.htm
- Advanced R: http://adv-r.had.co.nz/
R and its packages are free and open source. R is able to run across Windows, Mac, and Unix architectures. It is capable of efficient utilization of multicore platforms as well as large distributed architectures with thousands of nodes, providing leading data science scalability among technical computing languages. A recent HPC Wire story documents complex data science computations with pbdR, orders of magnitude faster than Spark's MLlib.
About the tutor(s)
George Ostrouchov is a Senior Scientist at the Oak Ridge National Laboratory and Joint Faculty Professor at the University of Tennessee. His research has been focused for many years on the interaction of statistics and high performance computing. Currently, he leads the pbdR project, recently noted on HPC wire. He is a fellow of the American Statistical Association.
The course is supported by The Ministry of Education, Youth and Sports from the programme Large Infrastructures for Research, Experimental Development and Innovations as part of the project „IT4Innovations National Supercomputing Center – LM2015070“.