Routing and Job Allocation in High Performance Clusters

Abstract

To achieve good performance in a High Performance Computing (HPC) environment we need to have in addition to a large amount of computing resources, a high performing network with high bandwidth and low latency interconnecting the computing nodes and storage nodes in the cluster, and a processor allocation algorithm that gives a good utilization of the computing resources and a high throughput of jobs.

The InfiniBand Architecture is a network technology that offers high throughput, low latency and support for multiple upper layer protocols. In the recent years, InfiniBand has increased its popularity in the HPC environment.

In this thesis we investigate both routing in InfiniBand networks and processor allocation.

In the first part of this thesis we studied, optimized and implemented the LAyered SHortest path (LASH) routing algorithm for the OpenFabrics InfiniBand software stack.

In the second part of this thesis we studied processor allocation strategies and introduced a new processor allocation strategy, the Spiral allocation strategy.