Abstract
Today's computing systems monitor and collect a large number of system load statistics, e.g., time series of CPU utilization, but utilization traces do not directly reflect application performance, e.g., response time and throughput. Indeed, resource utilization is the output of conventional performance evaluation approaches, such as queueing models and benchmarking, and often for a single application.
In this paper, we address the following research question: How to turn utilization traces from consolidated applications into estimates of application performance metrics? To such an end, we developed "Showstopper", a novel and light-weight benchmarking methodology and tool which orchestrates execution of multi-threaded benchmarks on a multi-core system in parallel, so that the CPU load follows utilization traces and application performance metrics can thus be estimated efficiently. To generate the desired loads, Showstopper alternates stopped and runnable states of multiple benchmarks in a distributed fashion, dynamically adjusting their duty cycles using feedback control mechanisms.
Our preliminary evaluation results show that Showstopper can sustain the target loads within 5% of error and obtain reliable throughput estimates for DaCapo benchmarks executed on Linux/x86-64 platforms.