How AlphaSense Cut CI Test Time by 2-3x While Scaling to 5,000 Tests Across 350 Repositories with Testkube

TLDR

• AlphaSense's AI platform team supports a large, distributed engineering organization spanning roughly 50 teams, around 1,000 services, and up to 200 releases per week.
• Jenkins-based testing depended on manual UI configuration that became outdated every time a Kubernetes cluster was recreated.
• A move to multi-cloud clusters on AWS, Google Cloud, and Azure made configuration-as-code non-negotiable. Testkube was the Kubernetes-native, GitOps-compatible fit.
• With Testkube, testing scaled from a single monorepo with ~3,000 tests to ~350 microservice repositories with ~5,000 tests, running about 1,000 test executions per day across all environments.
• Parallel sharding of E2E tests has already cut CI execution time by 2-3x, failed tests now block deployments, and Jenkins, the Moon browser grid, and custom Helm charts are gone.

About AlphaSense

AlphaSense is the leading AI market intelligence platform, helping business professionals make data-driven decisions with confidence. Behind the product is an engineering organization of more than 500 engineers across roughly 50 teams, maintaining around 1,000 services and shipping up to 200 releases per week at peak.

Valerii Timofeev leads the five-person platform team responsible for making all of that testable. His team does not write the tests. Their customers are AlphaSense's own developers and quality engineers, and the team owns the platform those engineers rely on: integration tests running in Kubernetes, 300 end-to-end Playwright tests, synthetic checks in production, and performance testing, spanning five frameworks in total: pytest, JUnit, Jest, k6, and Playwright.

"If tests cannot execute in a timely way, that delays releases," Valerii says. "We try to make a happy path to production, so engineers get their test results, video recordings, everything, and can make confident decisions."

The problem

For years, the execution layer for testing was Jenkins, triggered through webhooks after every deployment. It had one advantage: everyone knew it, down to junior manual QA engineers. But the cracks were structural.

Pipeline definitions lived in Groovy, and in Valerii's experience they always ended the same way. "Whatever project I saw Jenkins on, it always results in unreadable Groovy code," he says. The UI made things worse: "When the console log output is more than 20 megabytes and you try to open it in a browser, it just freezes. It has been that way since the beginning of my career."

The deeper issue was that configuration lived in the UI rather than in Git. Clusters had to be registered in Jenkins manually, and when a cluster was recreated, its configuration simply became outdated. Even with Groovy scripts in a repository, creating a pipeline still meant clicking through the UI.

Then the model broke down. AlphaSense began making its application self-hostable, standing up Kubernetes clusters across AWS, Google Cloud, and Azure. Those clusters were recreated or upgraded roughly every month, while engineering leadership was also committed to reducing platform tech debt.

Although Jenkins offered a Configuration as Code plugin, the team did not conduct a deep technical evaluation of whether it could manage cluster registration through the Jenkins Kubernetes plugin. Instead, they concluded that a Kubernetes-native approach offered a more direct path. Testing needed a new home, with one clear requirement: test orchestration had to be managed as code.

Discovering Testkube

The wish list was short and strict: Kubernetes-native, able to live alongside clusters that change and get recreated, and fully GitOps-compatible.

"At that point, there were no real Kubernetes-native alternatives built specifically for test execution," Valerii says. "There were general workflow orchestration tools, but we needed something designed for testing that could live alongside the cluster. We were already using GitOps, and Testkube fit directly into that model. That was the main selling point."

A Developer Experience engineer opened the first ticket, a DevOps colleague deployed Testkube Open Source with a Helm chart, and within two weeks Valerii owned the configuration. The team ran open source for about a year, even keeping Jenkins as a familiar front end while Testkube ran the tests underneath. "They opened the pipeline, which looks exactly the same. But under the hood we run tests on the Testkube engine," he says. When Jenkins was removed entirely, AlphaSense licensed the Testkube Control Plane.

Testing that blocks bad releases, every day

The results show up across the whole delivery pipeline.

Tests now gate deployments. In the Jenkins era, tests reported failures but did not stop anything. Today, if tests fail, the deploy is blocked, and Testkube's re-run capability keeps that policy workable. "Developers do not need to start from scratch. They run specifically this test, and it reports back to the original pipeline. If it passes, we allow the team to deploy," Valerii says.

Every integration test runs every day. Testing has grown alongside the architecture: what began as a single monorepo with around 3,000 tests is now roughly 350 microservice repositories with about 5,000 tests, spanning five frameworks (pytest, JUnit, Jest, k6, and Playwright). Testkube runs about 1,000 test executions per day across all environments, independent of deployments, so services that go months without a deploy no longer go dark. "Because we grab all of those tests and run them every day, we always know if something goes wrong. We know immediately, or within a day."

CI test time cut by 2-3x, with more headroom to go. Each test file runs in its own pod, and the E2E suite has evolved from a single shard to four to six parallel Playwright shards, each with its own browser. That made the Moon browser grid unnecessary.

Sharding alone has already reduced CI execution time by 2-3x, and the team's next goal is to scale to 50 parallel shards. They have already pushed parallelism far enough to trigger AlphaSense's own login rate limits.

"The limitation was not Testkube or the testing infrastructure," Valerii says. "We were reaching an application rate limit that was working as intended."

Performance tests that once failed with memory errors can now be scaled by adjusting CPU and memory parameters in configuration and rerunning them within seconds.

Visibility went from hours to a glance. Collecting test statuses across development, RC, and production pipelines used to take hours of manual work. "With Testkube there is a single Insights page and a single page for all executions. You can just open it and in a second get a general understanding of how things go."

Scaling with simplicity

The surprise win was ephemeral environments. Multiple teams share one backend repository at AlphaSense, constrained by a deployment pipeline that runs one deploy at a time. The platform team configured a Testkube Test Workflow that runs the service itself alongside the tests, so nobody waits for a deploy slot. "It drastically decreased the time they required to test things in Kubernetes. In YAML you specify your service name and test location, and it lives in the same UI as the other tests," Valerii says.

The team also carried one hard-won lesson into Testkube: everything comes from Git. Creating resources from the UI is disallowed entirely, so every workflow is tagged and owned. "We always know the history of anything created, and we always know who to go to if something is broken."

And testing now extends into AI-assisted development. "I tell Claude: you have access to the Testkube CLI. Please run this test multiple times with different parameters and tell me the output," Valerii says. "The agent gives me the report, and it can immediately fix the issue and push back to the branch."

The outcome

AlphaSense went from UI-bound Jenkins configuration that evaporated with every cluster change to a GitOps-managed testing platform that spans clusters on three clouds. Three tools were decommissioned along the way. Testing scaled from a single monorepo with 3,000 tests to 350 repositories with 5,000 tests across five frameworks, running about 1,000 executions a day. Tests moved from reporting failures to preventing them, parallel sharding cut CI execution time by 2-3x on the way to a 50-shard goal, and a testing setup that once took hours to reason about is now one Insights page.

‍