Your CI Pipeline Is Not a Testing Platform (and Why Feedback Takes 45 Minutes)

You push a commit. You open a PR. And then you wait.

The CI pipeline kicks off: build, lint, unit tests, maybe an integration suite if someone wired it up. Fifteen minutes pass. Twenty. You context-switch to something else. Forty-five minutes later, a red X appears next to your PR. You click through three layers of logs to find out a Selenium test timed out because the staging database was not seeded properly.

That is not a testing problem. That is an infrastructure problem disguised as a testing problem. And your CI pipeline, no matter how many YAML files you throw at it, was never designed to solve it.

Want the structural view? Why testing tightly coupled to CI/CD becomes a problem in Kubernetes environments. Read: The challenges of testing in your CI/CD pipeline →

CI pipelines are build tools, not test orchestrators

Is a CI pipeline the same as a testing platform? No. GitHub Actions, GitLab CI, Jenkins, and CircleCI are build tools. They are excellent at compiling code, running linters, and executing unit tests. They are not designed to orchestrate complex test suites across environments, manage test dependencies, or provide rich feedback on failures. A testing platform handles those concerns as a dedicated layer alongside CI/CD.

GitHub Actions, GitLab CI, Jenkins, CircleCI: these are excellent at building artifacts, running linters, and executing unit tests. They are designed to take code from commit to container. What they are not designed for is orchestrating complex test suites across multiple environments, managing test dependencies, or giving you meaningful feedback when something fails in a way that has nothing to do with your code.

The moment you start bolting on integration tests, load tests, or end-to-end suites to your CI pipeline, you are asking a build tool to do the job of a test platform. And the cracks show up fast.

Tests queue behind builds because runners are shared. Load tests cannot run at realistic scale because the CI runner does not have the resources. End-to-end tests fail because the environment they are hitting is not the environment your app actually runs in. And when something does fail, you get a wall of logs with no context about whether the failure was in your code, the test infrastructure, or the network between them. This is the tests pass locally, fail in CI environment mismatch pattern playing out at scale.

TomTom's B2C IAM team ran into exactly this. Before they restructured their testing, they were running tests on three dedicated virtual machines with no version tracking. Test scripts were manually triggered. Integrating testing frameworks into their CI/CD pipelines was complex. And visibility into test results across their development, testing, acceptance, and production environments was limited.

Iakym Lynnyk, an Expert Systems Engineer at TomTom, led the effort to change that. The problem was not that they lacked tests. They had SOAP UI tests for functional validation, JMeter for load and stress testing, and Selenium-based UI tests built with Maven. The problem was that none of these ran in a way that gave developers fast, reliable feedback.

The feedback loop tax

What does slow CI feedback actually cost a team? The clock time is the visible cost. The real cost is the context switch. When a 45-minute wait pulls a developer out of flow state, the value of the failure report drops by the time it arrives. By then they have moved to another task, lost the mental model of what they pushed, and debugging becomes archaeology.

Every minute your CI pipeline spends running tests is a minute you are not shipping. But the real cost is not the clock time. It is the context switch.

Studies on developer productivity consistently show that interruptions destroy flow state. When you push a commit and the feedback takes 45 minutes, you have already moved on to another task. By the time the failure notification arrives, you have lost the mental context of what you were working on. Debugging becomes archaeology: you are digging through code you wrote an hour ago, trying to remember why you made that change to the auth handler.

Multiply this across a team of ten developers, each pushing three to five times a day, and you are looking at hours of accumulated lost productivity. Not because the tests are slow, but because the infrastructure running them was not built for this purpose.

This is what buyers on real sales calls describe when they say: "Our CI pipelines are taking 45 minutes and developers are just getting coffee."

What changes when tests run as Kubernetes jobs

Why does running tests as Kubernetes jobs fix the feedback loop? Tests execute in the same cluster as the application, sharing the same network, DNS, and resource constraints. Feedback gets faster because there is no environment mismatch to debug, no shared CI runner queue, and no five-tab debugging path across CI logs, cluster logs, and app logs. Failures point at code, not infrastructure.

TomTom's solution was to move testing out of the CI pipeline and into the Kubernetes cluster where the application actually runs. Iakym spearheaded the migration from standalone virtual machines to a cloud-based Kubernetes platform, with Testkube integrated across all four environments: development, testing, acceptance, and production.

The shift happened in phases. Functional tests (SOAP UI) moved first, covering authentication and authorization. Performance tests (JMeter) followed, handling load, stress, and soak testing. Then Selenium-based UI tests, built in Java with Maven, were integrated for automated browser validation.

The key difference: tests now execute as Kubernetes jobs inside the cluster, not as steps in a CI pipeline running on a separate machine. That means the test and the application share the same network, the same DNS, the same resource constraints. When a test fails, you know it failed because of your code, not because of a mismatch between the CI runner and the cluster.

TomTom initially triggered tests via Azure DevOps pipelines, and as they migrated to GitHub Actions, Testkube triggers moved with them. The test definitions did not change. The execution context did not change. Only the trigger did.

See how TomTom automated testing in their CI/CD pipeline. Read the case study →

What developers actually get from this

What developer experience improvements come from moving tests out of CI? Four shifts. Faster signal on failures because debugging happens in one place. Tests that match production, which kills the "passes locally, fails in CI" pattern. Tool flexibility, because the orchestration layer is framework-agnostic. CI-provider independence, because the test definitions stay stable when you change CI tools.

Faster signal on what broke. When tests run inside the cluster, you eliminate the overhead of spinning up environments, configuring network access, and waiting for CI runners. TomTom integrated their test execution logs with a Prometheus stack, so engineers could trace issues from test execution directly to application logs. Troubleshooting time dropped because the debugging path went from "check the CI logs, then check the cluster logs, then check the app logs" to "check one place."

Tests that match production. The number one source of developer frustration with testing is not slow tests. It is unreliable tests. Tests that pass locally, fail in CI, and then pass again when someone re-runs the pipeline. That unreliability almost always comes from environment mismatch. When tests run in the same cluster as the app, the mismatch disappears. You stop wasting time investigating failures that are not real.

Use whatever tools you already know. TomTom runs SOAP UI, JMeter, and Selenium through the same orchestration layer. They did not have to abandon their existing test suites or learn a new framework. Testkube is framework-agnostic: you bring the test, it handles the execution, scheduling, and results. If you write k6 scripts, those work too. If you prefer Cypress or Playwright, same story.

Trigger tests from wherever you work. Whether your team uses GitHub Actions, GitLab CI, Azure DevOps, or Argo CD, you can trigger Testkube test runs from any of them. TomTom migrated from Azure DevOps to GitHub Actions mid-adoption and the test layer came along without rewriting anything. Your tests should not be locked to your CI provider.

The cost of not fixing this

What does it cost to leave testing inside the CI pipeline? Infrastructure cost on shared runners and dedicated VMs. Developer time lost to 45-minute feedback loops, re-runs, and context-switching. Hiring cost to staff QA engineers who manually trigger tests because the platform cannot. The visible cost is infrastructure; the invisible cost is engineering time across the team.

TomTom's Iakym was direct about the alternative: without a Kubernetes-native test platform, they would have continued running tests on expensive, untracked virtual machines with manual triggers and no centralized visibility. The cost was not just infrastructure (three VMs with no version tracking). It was developer time spent waiting, debugging environment issues, and context-switching.

The company is now exploring Testkube's commercial version for advanced parallelization and execution optimization, and advocating for company-wide adoption beyond the B2C IAM team. That trajectory, from one team solving their own pain to an organization-wide standard, is common. It starts with a developer who is tired of waiting 45 minutes for feedback.

Where to start

How do you start moving tests out of the CI pipeline? Pick your slowest test suite first. Run it as a Kubernetes job inside your cluster. Keep your CI pipeline as the trigger but move the execution into the cluster. Centralize results so failures debug in one place instead of five. Expand to other test types once the first one is trusted.

If your CI pipeline is your test platform and feedback takes longer than it should, here is the shortest path to changing that.

Identify your slowest test suite. That is the one causing the most context switches and the most "I'll just re-run it" moments. For TomTom, the functional tests (SOAP UI) moved first because they were the clearest win.

Run that suite as a Kubernetes job. Not as a CI step. As a job inside your cluster, where it shares the network and resources with the application. If you are already on Kubernetes, this is a smaller change than it sounds.

Connect the trigger to your existing CI/CD. You do not have to rip out GitHub Actions or GitLab CI. You just stop asking them to run tests directly and instead have them trigger test execution inside the cluster. The pipeline still kicks things off. The cluster does the heavy lifting.

Look at the results in one place. TomTom integrated Testkube with Prometheus for monitoring and log aggregation. The goal is simple: when a test fails, you should know why within minutes, not hours. One dashboard for results and artifacts, not five tabs across different tools. This is what centralized test observability looks like in practice.

Stop waiting for your pipeline

Your CI pipeline was built to ship code, not to tell you whether your code works. Those are different jobs, and they deserve different tools.

TomTom moved testing out of VMs and into Kubernetes, kept their existing frameworks (SOAP UI, JMeter, Selenium), and gave developers a faster feedback loop without rebuilding their CI/CD from scratch. As Iakym put it: "You are the only Kubernetes-native testing platform. Especially with testing workflows. That's what I love."

Testkube runs tests as native Kubernetes jobs, supports any framework, and gives developers instant feedback where their code actually lives.

Key takeaways

• CI pipelines are build tools, not test platforms. GitHub Actions, GitLab CI, Jenkins, and CircleCI are excellent at builds, linting, and unit tests. They were never designed to orchestrate integration, load, or end-to-end test suites at scale.
• The 45-minute feedback loop is an infrastructure problem. Tests queue behind builds, runners lack resources for realistic load, and failures cannot distinguish between code issues and environment issues. None of that is a "tests are slow" problem.
• The fix is not replacing CI. It is moving test execution into the cluster. CI/CD stays in place as the trigger. Tests run as Kubernetes jobs inside the cluster where the application lives.
• Environment parity kills false failures. Tests sharing the same network, DNS, and resources as the application stop producing the "passes locally, fails in CI" pattern that erodes trust in the suite.
• Start with your slowest test suite. TomTom moved functional tests first, then performance, then UI. The pattern is the same everywhere: pick the suite that causes the most pain, prove the model, then expand.

Frequently asked questions

Why is my CI pipeline so slow?

CI pipelines slow down when you ask them to do work they were not designed for. They are built to take code from commit to container: build, lint, unit test. Bolting on integration tests, load tests, and end-to-end suites turns a build tool into a test platform it was never architected to be. Tests queue behind builds, runners lack resources for realistic load, and feedback stretches to 30-45 minutes.

Is a CI pipeline the same as a testing platform?

No. CI pipelines like GitHub Actions, GitLab CI, Jenkins, and CircleCI are build tools. They are excellent at compiling code, running linters, and executing unit tests. They are not designed to orchestrate complex test suites across environments, manage test dependencies, or provide rich feedback on failures. A testing platform handles those concerns as a dedicated layer that sits alongside CI/CD.

Why does CI feedback take 45 minutes?

Three reasons. Tests queue behind builds because CI runners are shared. Load and end-to-end tests cannot run at realistic scale because runners lack resources. And when a test fails, you cannot tell whether the failure was in your code, the test infrastructure, or the network between, which forces re-runs and slow debugging.

How do I get faster developer feedback on tests?

Move tests out of the CI pipeline and into the same environment where the application runs. When tests execute as Kubernetes jobs inside the cluster, they share the same network, DNS, and resource constraints as the application. Feedback gets faster because there is no queue, no environment mismatch, and no five-tab debugging path across CI logs, cluster logs, and app logs.

Do I have to replace my CI pipeline to fix this?

No. CI pipelines like GitHub Actions, GitLab CI, Azure DevOps, and Jenkins stay in place to handle builds and deployments. They become the trigger source for tests, not the test runner itself. Tools like Testkube run tests as Kubernetes jobs inside your cluster, and your existing CI pipeline triggers them via webhook or plugin.

What is the difference between a CI pipeline and test orchestration?

A CI pipeline triggers and sequences events: commit, build, test, deploy. Test orchestration is the layer that handles execution, scheduling, parallelization, and results for tests specifically. CI/CD is about getting code from commit to production. Test orchestration is about validating that code at scale across environments. The two work together; they are not interchangeable.