Improving cloud performance by solving scalability limitations in libvirt

Abstract

At the heart of OpenStack, the worlds largest open cloud framework, lies libvirt. This C library is responsible for server consolidation, which again is the key to efficient utilization of the underlying hardware. However, preliminary experiments has showed that libvirt proved to scale poorly when handling large numbers of virtual machines. As libvirt is a core feature used within a multitude of applications, including several cloud services, it is essential that it scales well. Through this thesis the performance of libvirt is investigated through experiments, communication with the libvirt development team, profiling and as well as tests trying to isolate potential issues.

The issue turns out to be related to NUMA architectures, which is a hardware setup where different CPU cores have their own local memory. While this removes a potential bottleneck when it comes memory access, it introduces new complications for the software. Software used on machines with a NUMA architecture needs to be aware of the architecture in order to perform well. Upgrading software to be NUMA aware is an ongoing process with constant improvements.