Top 5 Challenges in Modern Kubernetes Testing
The shift to cloud native has left software testing woefully behind. Let’s improve your testing capabilities on Kubernetes.
The past few years have changed development beyond what many of us could imagine, and let’s face it, it’s mostly because of a little project called Kubernetes and the shift to cloud native. From containerization to microservices, we’ve embraced remote work, agile teams, and faster development and release cycles that cloud native enables us to manage.
But we’re missing a vital piece of the development cycle: testing. After all, when you’re deploying every day (or hour, or minute…), where’s the time for testing? Yes, it’s central to your product delivery and it’s imperative to get it right every time, but it’s like pulling teeth!
When we started working with Kubernetes, it quickly became apparent that there were major challenges with integration testing, especially when it comes to configuring tests in your continuous integration/continuous delivery (CI/CD) pipelines to follow a GitOps approach. So we built a framework called Testkube that makes testing easier with Kubernetes. We’ve specifically tackled more complex testing scenarios, where you might want to run tests against your UI, test different protocols or are running a microservice architecture with many services and versions.
Let’s look more closely at the top five challenges in no particular order that testers face in the cloud native space and how we can start making all our lives easier by adding in-cluster testing to your CI/CD pipeline.
1. Tight Coupling
There are a ton of advantages to tightly coupled architecture, especially if you’re working with large volumes of data and many sources. But it limits the freedom both developers and testers have over testing activities.
Because testing and test execution activities are closely coupled to your CI/CD and build workflows, you end up having to run your tests at the same time as your build. But what happens when you need to run tests that aren’t in sync with your build? Say you’ve updated a component and just want to rerun specific parts of your test suite? Or if you need to run a specific test when the orchestration is tied to CI/CD tools like GitHub Actions or Jenkins?
2. GitOps
The pressure of modern engineering! Don’t we all know it, the constant deadlines and cycles?
Thankfully GitOps gives us a single source of truth to work from so you’ll always know the state of your clusters and can work with them using well-established workflows. If you’re working with a mature DevOps approach, coupled with a solid GitOps framework, you can deploy an unbelievable amount of code to production each day. But where, and how exactly, does testing come in?
How do you tie your testing and test-related artifacts to the idea of managing all cluster states with git? Do you manage your tests in the same way? Apply them to each cluster? And how does testing fit into your GitOps CI/CD pipeline when it’s already happily chugging along churning out code?
3. Test Tool Diversification
It’s great that we can pick and choose our own languages and tools today, even individually within our teams. We get to pick the right tool for each job, and testing is no exception. We’ve seen teams using different testing tools for different purposes — API testing (SoapUI, Postman), end-to-end functional UI testing (Cypress, Selenium), load testing (JMeter, k6), not to mention homegrown frameworks for automation and integration testing.
The downside is that this results in different testing frameworks, tools and libraries that generate results in different formats. Some organizations even build a specific framework that works on one language for certain tests, which is wonderful until that one person from the team who knows how it works inevitably leaves and all that institutional knowledge is gone.
As a tester, you can’t be an expert in everything. But because testing touches so many parts of the stack, there needs to be a standardized method that’s easy to run and easy to monitor, no matter your language or tooling preferences.
4. Measuring and Monitoring
Ever had a sixth sense about why something’s going wrong in your build before you’ve seen the results? It’s easy to develop a nose for these things when your main focus is testing, but the growing asynchronicity of organizations is becoming more of a barrier here, like microservices managed by separate teams that could all have their own build pipelines. This asynchronicity is also unearthing issues with folks not understanding patterns in testing results, making it harder to detect when things are going in the wrong direction.
Consistently tracking metrics around QA and test pass/failure rates is important in organizations working with a huge number of different types of components and services. After all, without benchmarking, how can teams measure success?
5. Access Restrictions
We’ve all come across them — those pesky network access and security restrictions when deploying to Kubernetes, not to mention role-based access control, which can limit what you’re able to access or do within a cluster. And these restrictions are not easy to workaround. Of course, some of us are blessed with generous DevOps coworkers who will give you access as you need it, but that’s definitely not always the case. Plus, in the context of testing specifically, you might need cluster access to run your functional or performance tests that go well beyond what you would normally be granted.
Why Testkube?
We’ve experienced all of these frustrations — literally, all of them. We were tired of these limitations with cloud native testing, so we built a framework that explicitly gives developers and testers more freedom. We like to think of Testkube turning your cluster into a modern car. Modern cars run diagnostic tests on themselves to show the driver what’s wrong. Why not have your clusters do the same? Install Testkube and clusters can test themselves to reveal issues, without relying on any external tools, all while monitoring their own cluster health.
Testkube is a Kubernetes-native abstraction layer that allows testers and engineers to orchestrate and execute any tests without being dependent on a particular CI/CD tool. It manages the orchestration and execution of your tests from inside your cluster(s), meaning you can use as many CI/CD tools as you like and simply trigger tests to run. Plus, it takes the hassle out of packaging and configuring Docker by enabling testers to simply point to their collections/tests.
Let’s review why we believe TestKube is changing the game for cloud native testing:
1. Tight Coupling
- TestKube allows you to use any CI/CD tools so you can avoid vendor lock-in and stick to your favorite projects.
- You can uncouple your testing from your CI/CD build. Need to rerun a specific part of your test suite after updating a component? There’s no need to rerun tests for your entire build; simply pick and choose what you need and when.
2. GitOps
- TestKube integrates seamlessly with your GitOps pipeline and CI/CD workflow by allowing your tests to be version controlled while at the same time enabling them to be part of your cluster using Kubernetes custom resource definitions (CRDs).
3. Test Tool Diversification
- TestKube provides a standardized method that’s easy to run and easy to monitor, no matter your language or tooling preferences.
- Because Testkube is built on a modular architecture, it’s very straightforward to add features or support any testing tool that you see fit.
- Plus, there’s no Docker required.
4. Measuring and Monitoring
- With TestKube, you always know the state of your clusters. Testkube is aware of the definition of all your tests and respective results, so you can use it as a centralized place to monitor the pass/failure rate of your tests.
- Testkube defines a common result format, allowing for consistent result reporting and analysis across all types of tests.
5. Access Restrictions
- The Testkube server lives inside your cluster so you don’t have to configure any special network access since the test runs will run inside the cluster as well.
We’ve also just released Testkube Version 0.9, along with a bunch of useful new features, including a refreshed organization of tests, added as we continue to expand the executors we support, and we’ve made the TestKube dashboard even easier to read. So give it a spin and let us know what you think and how we can make it even better. Download the release from GitHub.
Do you agree with our top five challenges list? Please join our Discord server and follow us on Twitter @Testkube_io or email me directly bruno@kubeshop.io to let us know what other challenges you would like to see discussed next. We’re looking forward to hearing from you!
