The National Competence Centers in HPC of Turkey and the Czech Republic are inviting you to the jointly developed training: Mastering Transformers: From Building Blocks to Real-World Applications. 

Annotation

For the past five years, the amount of transformer-based architectures has grown significantly and continues to dominate the deep learning domain. They can be considered another leap innovation that further pushes the boundaries in deep neural network performance and scalability. They have been demonstrated with the most significant models using over half a trillion parameters and scaled up to thousands of GPUs.

In this course, participants learn the building blocks of transformer architectures to apply them to their projects. These novel methods will be differentiated against existing methods, showing their advantages and disadvantages. Different hands-on exercises give the participants room to explore how the transformers work in various fields of application.

Language

English

Level

Intermediate 

Prerequisites

Basic knowledge of Python. Knowledge of PyTorch is helpful but not mandatory. Basic principles of deep learning and machine learning.

Agenda

12:30 - 16:30 CET 

1:30- 5:30 PM GMT+3

1st half-day: Introduction to Transformers

  • Introduction and history 
  • DNN building blocks 
  • Advantages of Transformers 
  • Hands-On with PyTorch 
  • Machine Learning Lifecycle 
  • Q&A

 

2nd half-day: Transformers for Natural Language Processing (NLP)

  • Introduction to language models 
  • NLP Applications 
  • Hands-On: Build a simple transformer task 
  • Hands-On: Use (pre-trained) NLP models 
  • Q&A

 

3rd half-day: Transformers in action and their applications in other domains

  • Transformers in other domains 
  • Deployment 
  • Optimizations 
  • Multi-GPU/-node with PyTorch DDP 
  • Q&A

 

Tutors

Tugba Taskaya Temizel is a Professor in the Data Informatics Department at the Graduate School of Informatics of Middle East Technical University (METU). She finished her Ph.D. studies in the Department of Computer Science at the University of Surrey in 2006. She is the coordinator of METU IS Data Mining Research Lab. She is also the NVIDIA DLI Certified Instructor and DLI University Ambassador. She is certified to teach “Fundamentals of Computer Vision (CV) 2.0”, “Fundamentals of Multiple Data Types (MDT)”, “Fundamentals of Natural Language Processing (NLP)”; and “Building Transformer-Based Natural Language Processing Applications”. She was on sabbatical leave at the University of Birmingham, UK, between 2016 and 2017. Her research interests are deep learning, natural language processing, social media analytics, and data & information quality. 

Alptekin Temizel is a Professor at the Graduate School of Informatics at the Middle East Technical University (METU). He obtained his Ph.D. from the Center for Vision, Speech, and Signal Processing (CVSSP) at the University of Surrey in 2006. He is an Executive Board Member of the Neuroscience and Neurotechnology Center of Excellence and the METU Center for Image Analysis. He is the Principal Investigator of the Deep Learning and Computer Vision Research Lab and a Deep Learning Institute Certified Instructor and University Ambassador. Dr. Temizel frequently collaborates with and advises various companies. He was a Visiting Researcher at Microsoft MLDC-Lisbon during the summers of 2014 and 2015 and was on sabbatical leave at the University of Birmingham, UK, from 2016-2017. His primary research interests include computer vision, machine learning, deep learning, and GPU computing and optimisation.

Arif Ozan Kızıldağ  is a Research Assistant in the Information Systems Department at Graduate School of Informatics of Middle East Technical University (METU). He earned his bachelor's degree in Electrical and Electronics Engineering from METU and recently completed his master's thesis defense in the Information Systems Department. He is now preparing to pursue a doctoral degree in the same department. Throughout his academic career, Arif has also contributed as a Teaching Assistant for various deep learning courses within the department, such as "Deep Learning for Natural Language Processing" and "Foundations of Deep Learning." His research interests include deep learning, large language models, natural language processing, and prompt engineering.

Georg Zitzlsberger is a research specialist for Machine and Deep Learning at IT4Innovations. For over four years, NVIDIA has certified him as a University Ambassador of the NVIDIA Deep Learning Institute (DLI) program. This certification allows him to offer NVIDIA DLI courses to users of IT4Innovations' HPC services. In addition, in collaboration with Bayncore, he was a trainer for Intel HPC and AI workshops and conferences across Europe. He has contributed to these events for audiences from industry and academia for five years. Recently, he also received instructor certifications from Intel for oneAPI-related courses.

 

 

Acknowledgements

   

This course has received funding from the European High-Performance Computing Joint Undertaking (JU) under grant agreement No 101101903. The JU receives support from the Digital Europe Programme and Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Poland, Portugal, Romania, Slovenia, Spain, Sweden, France, Netherlands, Belgium, Luxembourg, Slovakia, Norway, Türkiye, Republic of North Macedonia, Iceland, Montenegro, Serbia. This project has received funding from the Ministry of Education, Youth and Sports of the Czech Republic.          

 

This course was supported by the Ministry of Education, Youth and Sports of the Czech Republic through the e-INFRA CZ (ID:90254).

Starts
Ends
Europe/Prague
[ONLINE]

This course is provided free of charge.

Please note that the training is held using Zoom. We advise all participants to download the Zoom application to enjoy full functionality. 

Only registered participants will receive the Zoom link.

Registered and confirmed participants will receive an account at an HPC infrastructure so that they are able to do the hands-on exercises, which are an integral part of the course.

Please note the time zones the course is held:

Sept 11-13, 2023, 12:30 - 16:30 CET Prague, Czech Republic

Sept 11-13, 2023 13:30 - 17:30 GMT+3 Ankara Turkey

The training consists of lectures and hands-on sessions. All registered participants will be able to follow the lectures; however, the hands-on sessions will have a limited capacity of 100 people. In case of more registrations, a brief form will be sent to the participants. Based on the submitted forms, participants will be selected by considering their experience, area of research, and qualifications. Participants should meet the qualification requirements of the hands-on sessions. Notification about acceptance will be sent within a few days before the event. Selected participants (for the hands-on sessions), who don’t have an account on TRUBA infrastructure, will receive a temporary user account for the hands-on training. 

 

Please note that the allocated compute time should only be used for running the hands-on examples. 

 

If you are unable to attend, please cancel your registration as soon as possible so that others may have the opportunity to benefit from the hands-on sessions.

 

Please note that a certificate of attendance will be given only to participants who will follow a minimum of 70% of the whole duration of the course and will be signed in with full name.