[ONLINE] Mastering Transformers: From Building Blocks to Real-World Applications (EuroCC)
[ONLINE]
The National Competence Centers in HPC of Turkey and the Czech Republic are inviting you to the jointly developed training: Mastering Transformers: From Building Blocks to Real-World Applications.
Annotation
For the past five years, the amount of transformer-based architectures has grown significantly and continues to dominate the deep learning domain. They can be considered another leap innovation that further pushes the boundaries in deep neural network performance and scalability. They have been demonstrated with the most significant models using over half a trillion parameters and scaled up to thousands of GPUs.
In this course, participants learn the building blocks of transformer architectures to apply them to their projects. These novel methods will be differentiated against existing methods, showing their advantages and disadvantages. Different hands-on exercises give the participants room to explore how the transformers work in various fields of application.
Language
English
Level
Intermediate
Prerequisites
Basic knowledge of Python. Knowledge of PyTorch is helpful but not mandatory. Basic principles of deep learning and machine learning.
Agenda
12:30 - 16:30 CET
1:30- 5:30 PM GMT+3
1st half-day: Introduction to Transformers
- Introduction and history
- DNN building blocks
- Advantages of Transformers
- Hands-On with PyTorch
- Machine Learning Lifecycle
- Q&A
2nd half-day: Transformers for Natural Language Processing (NLP)
- Introduction to language models
- NLP Applications
- Hands-On: Build a simple transformer task
- Hands-On: Use (pre-trained) NLP models
- Q&A
3rd half-day: Transformers in action and their applications in other domains
- Transformers in other domains
- Deployment
- Optimizations
- Multi-GPU/-node with PyTorch DDP
- Q&A
Tutors
Tugba Taskaya Temizel is a Professor in the Data Informatics Department at the Graduate School of Informatics of Middle East Technical University (METU). She finished her Ph.D. studies in the Department of Computer Science at the University of Surrey in 2006. She is the coordinator of METU IS Data Mining Research Lab. She is also the NVIDIA DLI Certified Instructor and DLI University Ambassador. She is certified to teach “Fundamentals of Computer Vision (CV) 2.0”, “Fundamentals of Multiple Data Types (MDT)”, “Fundamentals of Natural Language Processing (NLP)”; and “Building Transformer-Based Natural Language Processing Applications”. She was on sabbatical leave at the University of Birmingham, UK, between 2016 and 2017. Her research interests are deep learning, natural language processing, social media analytics, and data & information quality.
Alptekin Temizel is a Professor at the Graduate School of Informatics at the Middle East Technical University (METU). He obtained his Ph.D. from the Center for Vision, Speech, and Signal Processing (CVSSP) at the University of Surrey in 2006. He is an Executive Board Member of the Neuroscience and Neurotechnology Center of Excellence and the METU Center for Image Analysis. He is the Principal Investigator of the Deep Learning and Computer Vision Research Lab and a Deep Learning Institute Certified Instructor and University Ambassador. Dr. Temizel frequently collaborates with and advises various companies. He was a Visiting Researcher at Microsoft MLDC-Lisbon during the summers of 2014 and 2015 and was on sabbatical leave at the University of Birmingham, UK, from 2016-2017. His primary research interests include computer vision, machine learning, deep learning, and GPU computing and optimisation.
Arif Ozan Kızıldağ is a Research Assistant in the Information Systems Department at Graduate School of Informatics of Middle East Technical University (METU). He earned his bachelor's degree in Electrical and Electronics Engineering from METU and recently completed his master's thesis defense in the Information Systems Department. He is now preparing to pursue a doctoral degree in the same department. Throughout his academic career, Arif has also contributed as a Teaching Assistant for various deep learning courses within the department, such as "Deep Learning for Natural Language Processing" and "Foundations of Deep Learning." His research interests include deep learning, large language models, natural language processing, and prompt engineering.
Georg Zitzlsberger is a research specialist for Machine and Deep Learning at IT4Innovations. For over four years, NVIDIA has certified him as a University Ambassador of the NVIDIA Deep Learning Institute (DLI) program. This certification allows him to offer NVIDIA DLI courses to users of IT4Innovations' HPC services. In addition, in collaboration with Bayncore, he was a trainer for Intel HPC and AI workshops and conferences across Europe. He has contributed to these events for audiences from industry and academia for five years. Recently, he also received instructor certifications from Intel for oneAPI-related courses.
Acknowledgements
This course has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903. The JU receives support from the Digital Europe Programme and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg, Slovakia, Norway, Türkiye, Republic of North Macedonia, Iceland, Montenegro, Serbia. This project has received funding from the Ministry of Education, Youth and Sports of the Czech Republic.
This course was supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254).
