I often use the HPC cluster for the projects I’m working on. Recently we used the cluster to analyse around 2,000 genomes, identifying similar DNA sequences in different organisms – indicating that they have a common evolutionary ancestor. This generated roughly 1 million clusters of sequences, with each cluster needing to be analysed independently.

I also use the cluster for functional annotation of sequences, which requires the same type of massive parallel computation. The HPC cluster is perfect for this, because we need a lot of CPU time in independent nodes, and we need to run as many parallel jobs as possible.

If we have something like 1,000 jobs running at the same time and writing intensively to the disk, this can cause the file system to crash. So, if I need to run something like that, I usually get in contact with Jurij first. With him I can plan the best strategy to submit my jobs at any time, optimising the use of disk, CPU, and memory according to the current state of the cluster. For instance, we now have a very good system to automatically process data using scratch and temporary local disk areas, then move the results back to the shared file system. This has speeded up our computations a lot.

I’m very happy with the new system they’re setting up, using modules and Slurm, which is the state-of-the-art way of doing things in high-performance computing. It’s really appreciated by the people who are using these resources a lot, myself included.

Another thing I like here is having direct access to the admins, so if something is failing I can just message them straight away. We have a chat system with a channel for the HPC cluster, and the admins are very responsive. That’s very different to what you’d have if using external HPC resources, which don’t have such a personalised support system. In that case you’d have to apply for computing time in an external cluster, and then the support you’d get would be much more limited. Here I can design the jobs I’m submitting directly with Jurij, and ask what he recommends. Being able to do that is very cool.