Automated Load Testing in Kubernetes: How FunXtion Leverages Testkube

We are a company with a single office based in Amsterdam, and currently our Scrum team composition is our Product Owner, UI/UX Designer, five Software Engineers, two DevOps Engineers, and one QA Tester.

With most of our team being Developers, we have a few working closely with operational tasks, which leaves very few Engineers responsible for quality, that’s why test  automation is an essential part of our product development lifecycle.

The Background

Before Testkube, we had some of our traditional testing processes done within CI  pipelines, and therefore we lacked the ability to perform testing natively within our Kubernetes clusters. We were also relying heavily on manual QA testing, since there entirety of our scenarios weren't fully automated. The reporting and notification from QA processes were quite scattered as there were multiple places required to consult depending on the type of test executed. 

The Challenge: From Monolith to a Microservices Architecture

In 2022, we were moving from a monolithic application architecture running on premise infrastructure towards a public cloud infrastructure running completely in Kubernetes with a microservices-based architecture. 

In this context, we were also adopting a GitOps philosophy when implementing our infrastructure, QA, and release strategies - and we found that Testkube and its Helm Chart project fit  perfectly into our technological vision.  

The Solution: Testkube Adoption

We started using Testkube v1.6.72 around December 2022. 

One of the motivating factors in adopting Testkube was the new architecture which we were starting to move towards. 

In addition, we also wanted to implement our QA processes in a way that was friendlier to this type of modern architecture and to allow our QA engineers to work in what they do best: testing. Saving them from being too caught up in infrastructure or Kubernetes aspects.

The last factor for its adoption was the willingness of the teams to adopt something new which leverages automation, modern best practices, and the fact that the project is open source with a high frequency of contributions together with the awesome release notes done regularly. 

Our starting point with Testkube was to automate our API loading tests (making use of the k6 executor) to make sure that when making new releases we guarantee API performance metrics to be under specific threshold values. After that, we started to move our End-to-End tests into the cluster to fully exploit Testkube capabilities.

Currently, this allows us to leverage QA automation a bit more, be more agile and speed up the QA feedback-loop together with increased product quality.

The Impact

In adopting Testkube we acquired some of the following benefits: 

• Centralized reporting and logging of natively-run tests triggered automatically by Kubernetes deployments. 

• We decoupled some QA testing processes from CI pipelines since now we can run things natively within K8s clusters.

• We can define which QA tests to run in different environments in a declarative fashion following a GitOps philosophy. 

• API load testing is automated on deployment and visualization of results are displayed in a Grafana Dashboard, so we have a better assessment of performance aspects. 

Overall, it also allowed us to include our performance tests in our CI/CD process and keeps us reassured that we are not breaking quality metrics with new releases.

We don't have to keep track anymore on manual execution of performance tests by a few engineers with required setup and knowledge, but the process is now fully automated without extra costs on hardware and external systems. Now, we utilize the existing Kubernetes infrastructure to run tests inside the cluster when needed. We also have a strong confidence in test results as they’re running inside the development cluster, so there are no latency delays or network issues shown in the final test report.

The Future

I think in the future we will keep expanding with more native testing and potentially make use of other Testkube executors. 

Another interesting area that I believe we will explore more is the Test Observability aspect with Real-Time testing visibility. As our systems grow, we will increase the number of tests that will be running periodically or on demand for some user flows or system-to-system interactions.

We think that being able to define these tests, run and monitor their execution in an integrated platform is an interesting proposition from the QA and Site Reliability point of view.