Abstract
Software architecture practice relies more and more on data-driven decision-making. Data-driven decisions are taken either by humans or by software agents via analyzing streams of timeseries data coming from different running systems.
Since the quality of sensed data influences the analysis and subsequent decision-making, detecting data anomalies is an important and necessary part of any data analysis and data intelligence pipeline (such as those typically found in smart and self-adaptive systems). Although a number of data science libraries exist for timeseries anomaly detection, it is both time consuming and hard to plug realtime anomaly detection functionality in existing pipelines.
The problem lies with the boilerplate code that needs to be provided for common tasks such as data ingestion, data transformation and preprocessing, invoking of model re-training when needed, and persisting of identified anomalies so that they can be acted upon or further analysed. In response, we created a toolbox for realtime anomaly detection that automates the above common tasks and modularizes the anomaly detection process in a number of clearly defined components.
This serves as a plug-in solution for architecting and development of smart systems that have to adapt their behavior at runtime. In this paper, we describe the microservice architecture used by our toolbox and explain how to deploy it for obtaining an out-of-the-box solution for realtime anomaly detection out of ready-to-use components.
We also provide an initial assessment of its performance.