Introduction to High Performance Computing

In High Performance Computing we are interested in realizing numerical algorithms in such a way that we get as close as possible to the theoretical peak performance of the underlying hardware. This requires adapting the numerical algorithms. In the usual lectures on numerical analysis or numerical linear algebra this does not get covered.

It turns out that the performance only depends on a few elementary linear algebra operations. These operations became known as BLAS (Basic Linear Algebra Subroutines). So obviously an efficient BLAS implementation is crucial for scientific computing and scientific applications. Both, commercial ( Intel MKL, AMD ACML, Sun Performance Library, ...) and open source (BLIS, OpenBLAS, ATLAS, ...) implementations of BLAS are available.

So if a lecture is called High Performance Computing it has to deal with BLAS! One way of dealing with BLAS is merely reading papers and using existing BLAS implementations as black box. But if you really want to understand it you have to implement your own BLAS! In this lecture we implement some of the most relevant parts of BLAS (which we will call ulmBLAS) and it will be on par with commercial implementations.

How will we do this lecture online?

Due to the Corona situation we will do this lecture online. On the left-hand side you see a navigation bar with a list of sessions. Each session covers a certain topic of this lecture and is subdivided into a bunch of sections. In the first section I will outline the topic. In the subsequent sections I present you some video tutorials and eventually further information or exercises. It is important to understand that in general the content of these sections depend on each other. So work through these section in chronological order.

At least once a week I will offer an office hour through the BigBlueButton feature in Moodle. But of course, you also can reach me by email or the Moodel forum.

What are you supposed to do?

For each session start with with the first section and work your way through to the last session. That means read the text, watch the videos and simply reproduce what I show you. Ideally you even understand things that you reproduce ;-)

You only can learn programming (and this is the main content of this lecture) by doing things. This is a hands-on class and you have to make your hands dirty!

What should you do if something does not work?

Coding requires a high frustration tolerance. And you develop a high frustration tolerance by coding. But having said that, this is a lecture that helps you to get there. So do not hesitate to ask by using the forum or by sending me an email (did I mention that already?).

Syllabus and first lecture

The first virtual lecture is on April 21 from 4 pm to 7 pm. And this first lecture is covered in Session 1. Of course course you are free to start earlier but it is not required.

The schedule for the other can be found in the syllabus.

So what is session zero ?

For the first lecture we will use tools that run in the browser. So this should work on your machine. However, for later lectures we have to use tools that are only available on our Unix server. In Session 0 you will find help for connecting and using the server.

Actually it would be possible to install them on any Unix system (e.g. MacOS, Linux, etc.). But I never used a Windows machine in my long entire life, so I could not support that. Long story short: Using the departments Unix server is the easiest way to find some common ground for us.

Consider, that more time for the exercise will be required if you are not familiar with the following:

  • Working with a Unix terminal and

  • using a proper text editor (like Vim) for coding.