Initial Experiments with Duet Benchmarking: Performance Testing Interference in the Cloud

Abstrakt

Accurate performance testing may require many measurements and therefore many machines to execute on. When many machines are needed, the cloud offers a tempting solution, however, measurements conducted in the cloud are generally considered unstable.

In the context of comparing performance of two workloads, we propose a measurement procedure that improves accuracy by executing the workloads concurrently and using the measurements to filter outside interference. Depending on the platform used, experiments show average accuracy improvement ranging from 114% to 683% over sequential measurements on workloads running the ScalaBench suite with the Graal compiler.

Klíčová slova

• performance evaluation
• duet benchmarking
• cloud
• performance measurement