Extend Performance Assurance into Production Using Application Performance Management and Closed-Loop Testing

 Application Performance Management (APM) from the end-user perspective is not load testing per se, but it bears mention due to its close relationship to load testing. Load testing is used in a pre-production environment to predict an application’s performance, scalability and reliability using simulated loads before the application is deployed. APM is the counterpart of load testing, once the application is deployed into production. APM measures application performance, availability and accuracy under actual loads in the real world, and provides diagnostic tools to proactively identify and quickly resolve problems once they occur.

 As pre-production load testing and APM have a lot in common, assets developed in earlier load testing phases should be reusable later in production. Similarly, feedback (errors, information, etc.) from production should be able to be fed back into pre-production for load tests that reflect real-world usage conditions. This process is called “closed-loop testing” and is shown in the figure

Closed-loop testing not only provides a way to leverage already existing test assets used in preproduction testing for APM, but also for transferring knowledge between R&D, QA and Production. When the feedback loop drives production assets back into QA, it also allows for the validation of load testing results through real-life usage data. Furthermore, baseline metrics derived from pre-deployment load testing can be used as threshold values that proactively flag a potential performance problem once the application is deployed.

 Additionally, the closed-loop approach increases load testing efficiency. Performance metrics measured in production can become performance standards for the next software release cycle to meet. Captured input values from real-users can be used as test data for realistic data-driven testing, moving tests closer and closer to real-world usage scenarios. The result is better test coverage, more realistic test scenarios and more accurate performance metrics. Recorded sessions of actual users encountering an error situation can be fed directly back into pre-deployment and used as load testing scripts, cutting time needed for recreating - and resolving - an error situation.