Stop Running Tests in CI | PlatformCon 2026

Introduction

Hey there, welcome to this session on why you should stop running tests only in CI/CD at PlatformCon 2026. I'm Ole Lensmar, CTO and co-founder of Testkube, and I'm going to talk to you about this fascinating subject in a short time, so let's jump straight in. First of all, testing today happens everywhere across the pipeline, right? It's really not confined to CI and or CD.

So, testing is part of the planning phase, it's part of when you're writing code, you run tests locally, you might even have local environments that you're running like into integration tests in. As part of your builds, you usually automate some unit tests or component tests. Then, if you have a testing phase in CI/CD, that's obviously where you might be running your tests. As part of your release process, you might be running tests to validate the release as it's going out.

Once it's been deployed, I'm sure you're going to do testing in your pre-production or staging environments, and also in operations and production, where we also see a lot of people actually running tests just to as kind of an online monitoring solution. So, and as we've been shifting left for quite some time now, testing happens earlier, and as we've been shifting right lately, I guess, testing happens later in the pipeline. And of course, now we have AI kind of just fueling all of this, both kind of adding to the code and to the test that we're generating, which can of course result in slower pipelines. It's also something that testing needs to accommodate to.

Why testing needs to be continuous

So, testing really needs to be continuous, right? It has to happen at every stage of the software development life cycle. It has to be triggered automatically for different events along your pipeline, either by infrastructure events, or build events, or deploy events. You need observability in your testing.

You need to see you know the logs and results and artifacts and metrics and whatever else you can get to really understand the results of your tests. And you need to use tests to enforce quality gates, right? As you promote releases maybe from a testing to a staging to a release production environment. Gating those releases in an automated way based on test quality is essential.

And we have this beautiful Venn diagram to kind of reinforce that.

The challenges of continuous testing

So, there's obviously there's not that is just as easy as it sounds. There's a bunch of hard challenges with all of this. Let's talk through some of those. First of all, it's just consistency.

If you're running tests locally on your machine and you're running them in your build pipeline and you're running them in your production or pre-production environment, you want to be sure that you get the same results everywhere, right? So, you don't want the works on my machine or works in my pipeline. You might be using multiple CICD tools. So, consistency can definitely be a challenge because those different places might be using different environments, different runtimes, different versions of your testing tools, different infrastructure resources, etc.

Next one is on observability, especially with cloud-native applications that are highly distributed. You need to get insight in all the logs and the results and the metrics and the artifacts and whatever else that kind of is part of the outcome of your tests, especially if they failed so you can kind of troubleshoot and do root cause analysis. And that can be a challenge in a in a distributed environment. Promoting releases is the next challenge, right?

How do you in an when you're using GitOps or an asynchronous delivery process, how do you promote releases? How do you know when you're ready to put your new version of your app in production, especially if your if your app is consist of many microservices that maybe are evolving together? And what kind of criteria do you set up? Not just functional criteria, but also non-functional criteria around security or performance or other aspects.

Finally, how do you test the infrastructure itself? All right, if you're provisioning clusters, maybe ephemerally as part of your delivery pipelines or your platform team that builds clusters for application teams as they build new services or features, you want to make sure that the infrastructure you provision is a valid, right? It has the right that it needs, it has the resources it needs, etc. It's so infrastructure testing is becoming key aspect of testing and quality itself, pre-product pre-application testing or functional testing.

Finally, platform and well, not finally, as you see, but platform enablement is also super important important. Platform teams that want to make sure that the the developer teams in their organizations are consistent with how they test, how they collect their results, how they create their quality gates, but they also need a certain level of freedom, right? Which tools to use, maybe one team wants to use one end-to-end testing tool, another team might want to use another. So, how do you kind of enable them with certain constraints and guardrails while still having a kind of cement aggregated approach that can give you maybe the inch metrics that you need to judge quality across all your teams or in within your organization.

And finally, AI obviously is driving the generation of code, of configurations, of tests, and you have AI agents that might need to be tested in themselves. How do you kind of factor that all into this whole continuous testing idea or challenge that you have as tests kind of proliferate across the type pipeline and which are often driven by AI itself?

A centralized approach to testing

So, wouldn't it be great then if you could launch any test anytime in any environment from one place, if you could have your consistent test results, if you could have a unified view of all your testing activities, if you could know kind of these teams are running these tests pre-production in their local environments in their pre-production post post deployment post building environments. And you can also self-enable teams to test consistently and you can leverage AI to fuel your pipelines and not just clog them with more stuff that's kind of slowing down the delivery process. And that's really the challenge that problem that we tried to set out to solve with Testkube, which is a project that I'm going to talk a little bit about and how that kind of ties into all of these challenges that I just talked about.

What is Testkube?

So, first of all, what is Testkube? If you haven't heard of it, now you have, I'm glad. It's a test orchestration platform for Kubernetes that decouples test execution from CI/CD, meaning you through Testkube you can run any kind of test. It's vendor tool agnostic whenever you need across your delivery pipelines like all those arrows we saw arrows we saw in that initial slide.

It then aggregates and centralizes all your results into one place. It's GitOps ready, it uses Kubernetes constructs at its core like CRDs and pods and jobs and all the kind of stuff. I'll get back to that. So, it's really easy to integrate into your existing ecosystem of cloud native or Kubernetes tooling.

It has an open source tier obviously or not obviously, but it does. So, you can definitely install the Testkube open source agent, which I'll get back to in just a little while, in your infrastructure to leverage many of the features that going to be talking about or that or to solve many of the problems that I alluded to with continuous testing earlier. It works totally in an on-prem air gapped deployment. You can run this in your clusters both the the open source and the commercial and and very constricted or constrained environments.

So, it works really in all kinds of setups.

What Testkube doesn't do

I'll also point out what it doesn't do. Testkube is not a testing tool. It doesn't kind of replace your existing tests. It actually leverages your existing tests.

So, if you have a bunch of Playwright or Cypress or k6 or Postman or Pytest or I don't know, God knows what kind of testing you're already doing. Testkube basically takes those tests and runs them for you inside your Kubernetes infrastructure. So, it's not ask We're not asking you to rewrite your tests, but we're giving you a capability to run them in a way that solves many of the problems that I alluded to regarding continuous testing earlier on. It doesn't support device testing cuz devices are really hard to simulate in Kubernetes.

Which you can simulate Android devices to some extent, so that's actually something that Testkube can do, but if you need to simulate hardware devices or just iOS, for example, you want to test a mobile app, you'll have to connect Testkube to a device farm, and there are both commercial and non-commercial vendors that will provide that for you. And it also doesn't do testing environments. So, Testkube is not the tool that's going to stand up your cluster or your ephemeral namespace where you deploy your app to run your tests. And there are great tools out there like Okteto, you can use a vcluster, or even Argo itself can kind of create ephemeral namespaces when deploying.

So, that's something you would use to provision an environment, and then Testkube would run those tests for you.

Running tests inside Kubernetes

Testkube, as I mentioned, leverages Kubernetes for test execution. So, with Testkube, tests run inside your cluster alongside your services and apps, which means that you don't need to open up external access. Let's say you're testing microservices that aren't externally accessible. Since it's running in basically in Kubernetes, it can run any testing tool or script or test that you have as long as you can containerize it.

So, which most testing tools today support, especially the open source ones. Also, many of the commercial testing tools out there have Docker runners, etc. So, we can really just you can really run any kind of test. So, we even have chaos testing and infrastructure testing, which I'll get back to in a little while, which is a very, very common use case.

It Since you're running in Kubernetes, it can leverage the scalability of Kubernetes itself and those features to really run massive scale at massive scale. So, you can run your k6 tests across 100 pods, you know, generating massive load if you need to shard out your Playwright or Cypress end-to-end tests across multiple nodes to speed up execution. Testkube has primitives in its abstraction layer to support that. So, it's really built for test execution at scale and leveraging Kubernetes capabilities to do that.

It enables infrastructure validation, which is a very common use case. Validating infrastructure that is kind of provisioned ephemerally or more long-lived, and this could be validation that valid you know, validates resources. Maybe you need to make sure there's some GPUs available for AI inference workloads, or memory, which is usually needed as well. You might want to validate that you have network topology and constraints in place.

It could be anything kind of really that you need. And since Testkube is so close ingrained in Kubernetes, you have very close control over the tests that you run, how they leverage and tie into Kubernetes constructs. So, in a much tighter fine more fine-grained way than you would have, for example, when using a GitHub runner in Kubernetes or a Jenkins runner. So, there's definitely if you need to tap into the low-level qualities or features of Kubernetes, you could do it.

And you can do all of this with the open-source version of Testkube. This is not something that's somehow constrained to the commercial version.

Testkube architecture

At a high level, architecturally, Testkube has an agent that you install. It's the bottom part here. The agent is installed in your namespaces. It'll run your tests as I mentioned earlier.

It contains the actual execution engine, which is kind of what runs your tests in Kubernetes. And as I said before, it's open source, you can go to GitHub, grab it, install it with the CLI or the helm chart, and you can do many of the things that I mentioned before. The agent is managed through a CLI or a REST API. If you want to orchestrate multiple agents to do multi-cluster test orchestration, or if you want a nice dashboard where you have RBAC, AI, reporting, all those kind of features, that's where we have a control plane, which is also something you can install in your Kubernetes infrastructure.

This is a commercial component that provides many enterprise related features both on the functional side, but also on the operational side when it comes to security, etc.

Testkube in GitOps pipelines

Just to kind of share how does Testkube work in a common GitOps pipelines, right? So, if you're doing GitOps, you can definitely set up Testkube to do pre-sync validation. So, before your GitOps tool does a sync with you know, synchronizing the state of Git with your cluster, you can set up those tools if you're using Flux or Argo to validate that you know, your cluster is ready or your name space is ready for those synchronizations. You can do run tests while you're syncing, or you can sandwich is maybe the most common that post-sync like when Argo has updated and synced its application, you can configure it to run some tests for you to validate.

So, this is done outside totally outside kind of CI traditional CI. It's done more of as part of your GitOps deployment pipelines.

Enabling platform engineering teams

For platform engineering, Testkube empowers individual teams to kind of choose whatever testing tool they need, right? Which tools they use, how to trigger those tools in their pipelines. Some teams might want to run tests on a schedule. Other teams might want to run their tests like I mentioned earlier with GitOps.

Others might want to trigger them just manually or from of course their CI/CD tool, right? So, Testkube definitely allows you to continue triggering tests from, you know, GitHub Actions or whatever else. And using Testkube as a foundation for your platform quality platform allows gives your team the freedom to do all of that and how to do reporting and of course set up access controls. And then platform engineering teams can then leverage Testkube constructs to for consistency.

So, we have something called workflow templates to make you where you can ensure that all the teams are running tests in a specific way. We have something called webhook templates which allow you to manage that teams integrate with other ecosystem components consistently. And of course there's RBAC for access control.

AI capabilities

Obviously, there is AI functionality. Really quickly, we have AI agents that can autonomously triage and categorize failed tests. They can correlate test results with failures and flakiness. They can select and optimize which tests to run and they can analyze and optimize test execution.

All to kind of reduce resolution times and unblock or unblock your pipelines. And there's of course an MCP server that you can integrate with your existing tools. And finally, there is a support for bring your own MCP servers and LLMs. So, this can once again run totally in an air-gapped internal environment.

You don't have to use frontier or public models to tap into any of these features.

Key takeaways

Okay, I'm done here. Key takeaways, testing is continuous across the entire SDLC. AI is driving quality bottlenecks. A implementing a centralized and holistic approach to testing is key and Kubernetes provides a solid technical foundation for solving many of these continuous testing challenges.

And Testkube has you covered on all of these. So, that's it. Thanks for listening. Enjoy the conference and hopefully this was helpful.

Thank you.