Automating the TinyML Pipeline: From Model Compression to Edge Deployment

Abstract

Deploying machine learning on resource-constrained devices such as microcontrollers, especially in harsh environments (e.g., the arctic) presents significant challenges. This thesis explores solving these challenges in the framework of TinyMLOps, focusing on enabling live model updates and predicting inference latency on STM micro controllers. A method for seamless runtime weight updates via direct memory modification was implemented and we developed an automated workflow using the STM32EdgeAI REST API for benchmarking and code generation. Experiments confirmed successful live updates and achieved high accuracy in predicting single-layer network latency based on architecture (R2 > 0.98) and multi-layer networks up to 3 layers (R2 > 0.89). While predicting multi-layer network (more than 3 layers) latency requires further refinement, this work provides practical contributions to TinyMLOps, facilitating remote maintenance and establishing a foundation for hardware-aware model optimisation on edge devices.