MxKernel: A Bare-Metal Runtime System for Database Operations on Heterogeneous Many-Core Hardware

PIs: Olaf Spinczyk (Uni Osnabrück), Jens Teubner (TU Dortmund)
Project Collaborator: Hendrik Borghorst, Jan Mühlig, Michael Müller
Project website: mxkernel.org

New application classes are putting immense pressure on system software to process data faster and at larger scale. “Real-time analytics” demands high transaction rates concurrent to complex analysis queries; “Industry 4.0” or “Internet of Things (IoT)” applications challenge database and operating systems to process data distributed, scalable, and in real time.

Emerging hardware platforms can provide the necessary base performance. But leveraging it has become difficult; it is still largely unclear how large degrees of parallelism, complex memory hierarchies, or increasing hardware heterogeneity can be adequately supported by systems and application software. Clearly, different pieces of systems software must collaborate much more closely than what is available today.

We propose to build a bare-metal runtime system, which we call MxKernel. MxKernel provides very lightweight resource management for database and operating system, which both run as equal peers on top of the runtime. In MxKernel, heterogeneity and parallelism become first-class citizens, ready also for memory hierarchies or modern storage technologies (such as non- volatile memories). Instead of a classical “thread” model, MxKernel provides MxTasks as an abstraction for work items. MxTasks describe closed units of work, for which MxKernel will guarantee atomic execution.

MxTasks resemble “events” in event-based systems and inherit their excellent scalability as well as the potential to scale out to large, distributed systems. But MxTasks can be a very elegant abstraction also for heterogeneity and resource sharing. In the MxKernel project, we will enrich MxTasks with meta data, such as code variants (to support heterogeneity); memory access behavior (to improve cache efficiency and support memory hierarchies); or dependencies between MxTasks (to improve scheduling and avoid synchronization cost). With precisely the required meta data available, MxKernel can provide a lightweight, yet highly efficient form of resource management, even across applications, operating system, and database—a characteristic that we see increasingly important, e.g., in the light of cloud- based or consolidated systems.

In the project, we will investigate the MxKernel concept from both an operating system and a database system angle. The goal is to understand the feasibility of the concept; its implications on the design of (database) software; as well as its potential for performance and scalability. We will develop methods and guidelines to realize database tasks using the MxTask concept, including concrete solutions for critical database tasks (indexing and transaction processing).

The MxKernel project will be driven by the development of demonstrators. Building on conceptual results developed in the project, we want to quickly push our ideas to working prototypes, which will be modelling relevant and practical use cases from the transaction processing and IoT domains.