A before B: Investigations into how best to perform Temporal Health Queries

Abstract

Querying and exploring health data can lead to the discovery of new rela- tions between conditions, medications, hospital events, etc. For this purpose, temporal health queries are useful since the order in which events happen is important. Many of the querying tools available do not address the unique needs of temporal health queries, making these queries difficult and time-consuming to perform. One tool made for this purpose, Snotra, enables temporal health queries with a syntax that is human-readable and easy to understand and write. Problems in the technical implementation and underlying architecture of Snotra currently prevent it from being used to query large datasets from health registers. By implementing a subset of Snotra operations we can compare to design a new underlying engine for Snotra to handle larger datasets. This thesis ex- plores possible avenues to fix the underlying architecture of Snotra, comparing a selection of approaches including SQL, Dataframes, and custom low-level querying functions. The most promising approach is further developed into a prototype supporting a small subset of Snotra operations. This work shows how Polars extended with custom Rust query functions is a viable path for implementing performant and scalable temporal health queries.