Scaling kernel-based learning for big data

Abstract

Kernel methods are popular due to their solid and well-understood theoretical foundation. However, kernel-based learning methods generally have considerable memory and computational requirements that scale poorly with the number of training samples. Consequently, in real-world applications, these methods have largely fallen out of favor in the machine learning community with the rise of alternative methods such as multi-layer neural networks. This thesis focuses on developing algorithms that improve kernel methods' memory and computational requirements. The main strategy is creating kernel methods compatible with modern computational models such as parallelization, distribution, and streaming. We develop a novel multi-resolution learning scheme for streaming data (StreaMRAK) and demonstrate it in a biomedical setting. Furthermore, we develop an efficient kernel method for non-linear dimensionality reduction and improve existing graph metrics, extending them to the large graph regime. We validate the suitability of the algorithms we develop with numerical experiments, theoretical analysis, and comparison to existing methods in the field.