When delivering software to the market faster, there is a critical need to onboard automated tests into your continuous delivery pipeline to verify the software adheres to the standards your customer expects. Your continuous delivery pipeline could also consist of many stages that should trigger these automated tests to verify defined quality gates before the software can move to the next stage and eventually into production (see Figure 1). Depending on the stage of your pipeline, your automated tests could range in complexity from unit, integration, end-to-end, and performance tests. When considering the quantity and complexity of tests, along with the possibility of having multiple stages in your pipeline, there could be many challenges when onboarding, executing, and evaluating the quality of your software before it is released.
This article will describe some of these challenges. I will also provide some best practice guidelines on how your automated tests could follow a contract to help increase the delivery of your software to the market faster while maintaining quality. Following a contract helps to onboard your tests in a timely and more efficient manner. This also helps when others in your organization might need to troubleshoot issues in the pipeline.