September Release: The Test Your AI Wrote Isn't Testing Anything

Lucio and I recorded a demo that shows everything this post explains. If you'd rather watch than read, it's right here. No hard feelings either way.

Every team has the list.

The API tests you'd write if the sprint ever had slack in it. The load test that's been "next quarter" for three quarters. The flow everyone knows is fragile and nobody has covered, because the two people who really know your Playwright setup are busy.

Nobody disputes the list. It just never gets shorter. And lately it's been getting longer, faster, because your engineers are shipping more code than ever. A lot of it written with AI, in minutes, while every test still costs what it cost in 2020: someone's afternoon, plus the wiring, plus the repo, plus the workflow, plus actually running the thing against something real.

That's the gap. Code got fast. Test creation didn't. When we ask customers what they need from us, this is the answer we hear most, in almost the same words every time: help us create more tests.

Today we're releasing AI Test Creation in Testkube. It exists to make that list shrink.

Describe the test. Watch it run.

Here's what it looks like in practice.

You describe what you want to test, in plain language. "Create a Playwright test for the checkout endpoint and run it in staging." Any framework your team uses, and any kind of test: end-to-end, API, load, infrastructure. We built specialized skills for the most popular testing tools, so the system isn't reasoning about how a k6 script should look from first principles. It already knows.

Then comes the part that matters most: the test runs. Not "here's a file, good luck." It executes in your real infrastructure, seconds after it's generated. A load test runs in staging. A monitor runs against prod. You watch it pass or fail against something real while it's still a draft in front of you.

And when you accept it, it becomes yours in the plainest sense. The test comes back to your GitHub repository as a pull request. You review it like any other code, edit what you want, version it with the rest of your suite. No test locked inside our platform, no proprietary format, nothing you can't walk away with.

Describe, run, keep. The test that's been on the list for three quarters takes about as long as reading this section.

Wait, can't a coding assistant do this?

You're already thinking it, so let's have the conversation.

Cursor can write tests. Copilot can write tests. Claude can write tests. If generating a test file were the hard part, the coverage problem would already be solved, because those tools have been on your team's laptops for two years. The list didn't shrink.

Here's why. A test is almost nothing except specifics about your system: the real service name, the endpoint as deployed, what the response payload looks like, what else is running next to it in staging. A general model knows none of that, so it fills the gaps with plausible guesses. Then the file lands in your lap, and the real work starts: wire it into a repo, into a workflow, into an environment, and find out which guesses were wrong. Either it breaks for reasons that have nothing to do with your app, or, quieter and worse, it passes while asserting against its own assumptions. Your coverage number goes up. Your protection doesn't.

I want to be straight about the difference, because it isn't a smarter model, and I'm not going to pretend it is. It's three things a chat window doesn't have: skills built for how good tests get written in your framework, your actual infrastructure to run in so every test is proven against something real before you accept it, and a place for the test to live afterward. Same intelligence. Different inputs, and a runtime. That's the difference between a test file and a test.

The test doesn't land in a silo

One more thing, because it matters more than it sounds.

A test created this way isn't a loose file with nowhere to go. It lands in the same platform that orchestrates everything else you run. It executes in parallel with the rest of your suite, gets triggered the way your other tests do, and its results show up in the same place, next to everything else, where your team already looks.

Creating the test was the bottleneck. It isn't anymore. Everything downstream (running it at scale, watching it over time, knowing what failed and why) was already there.

Where this goes next

What I've described is what you get today. It's the first release of an engine we'll keep improving through Q4: generation that draws on your workflows, your environments, and your execution history, so the tests get better the longer Testkube runs your testing. We'll show some of that, live, in our webinar on October 8.

Go shrink the list

AI Test Creation is available now, included in the Testkube trial. Sign up, pick the test that's been on your list the longest, describe it, and watch it run in your own infrastructure a few seconds later.