QA Is Dead: Trust, Spec Recovery, and Autonomous Testing with JDAQA’s Jay Aigner

Meet Jay Aigner and JDAQA

Ole Lensmar (00:00) Hello everyone, welcome to another episode of the Cloud Native Testing Podcast. I'm super honored to be joined by superstar Jay Aigner. Jay, tell us what you're up to. What do you do?

Jay Aigner (00:10) I don't even know if I want to say the name of my company with the hat that I'm wearing, but I am the CEO of JDAQA, and we are a software quality assurance firm. I don't know what we are in 2026. Why don't you tell me what you think I am?

Ole Lensmar (00:23) Well, let me allude to the cap that you're wearing. Those of you watching this on video will see that it says "QA is dead." So that does go at odds with what you just said about what you're doing. I've obviously seen a lot about "QA is dead" on LinkedIn and other channels, with debates going back and forth. And you and I met in Philadelphia a couple of weeks ago and had an amazing event there with peers from the local scene. For those who are not versed in the "QA is dead" discourse, please, Jay, give us an intro.

Jay Aigner (00:54) You want me to annoy some more people. That's what you want me to do. I've made so many friends. Yeah, I mean, that's kind of how I've always done it, I guess.

Ole Lensmar (01:00) It's the best way to make friends, to annoy them. And then one leads to the other.

What does "QA is dead" actually mean?

Jay Aigner (01:06) So, early this spring, we felt the market had shifted. What I always say is that QA has always been broken. The concept of this squishy thing at the end that catches all the stuff, where some stuff gets out to production and those guys get yelled at — the whole process is broken. That worked for 20 or 30 years, but at the speed code is coming out now, it literally does not work anymore. So "QA is dead" means that you can't just bolt it onto the end and hope you catch everything. You really need to understand, across product, engineering and validation, how the product is being moved through that pipeline, because the end results are now more affected than ever by the quality of what you put into the machine. Hopefully that answers the question.

Is the QA community embracing autonomous quality?

Ole Lensmar (01:53) It definitely leads us down a path, so let's explore that. Maybe I'm late to the game here, but what was eye-opening for me at the event we did — and I was at LeadDev New York the week before and at StarWEST the week after — is how far teams and orgs are pushing this concept of autonomous quality. We can debate that, but I was surprised at StarWEST, which is a QA conference where historically there's been a lot of skepticism around tooling and automation. I was expecting skepticism and pushback on AI, but there was very little. Most people were saying, "Yeah, we're building AI into our quality pipelines," manual testers are using AI to generate their test plans, and all the vendors are talking about autonomous quality loops and feedback. So it does seem to me that there's this big hope or dream that AI has conquered coding and now AI is conquering QA. It sounds nice, but of course it's easier said than done. I'm wondering what nails people are going to step on along that path, how they'll learn, and where we'll end up. What's your take, and what have you seen in the space?

The trust problem: would you run it on a helicopter?

Jay Aigner (03:07) I'll answer your question with a question. Was there anything at StarWEST where you thought, "I could run that on a mission-critical piece of software where people's lives are on the line"? Where you'd trust it, run it, and believe that the lights on the helicopter the software controls actually do what they're supposed to do, nobody dies and everything's great? Did that exist at StarWEST?

Ole Lensmar (03:32) Not that I saw. But what did exist at StarWEST is the drive to get there — the urge to find a way to verify software autonomously, to have enough confidence to release it and control that helicopter. We're not there yet, obviously, and building confidence is the hard part. But it doesn't feel like people are pushing back on the concept. It's just that we haven't really figured out the prompts, the harnesses and the pipelines yet. We'll get there. Maybe we're overly optimistic, based on the evolution so far.

After the AI scare: why a real 4.5X beats 10X

Jay Aigner (04:05) No, I think you're right. I've seen that shift on social media, even on LinkedIn, in the past three months since we started doing "QA is dead." I would love to think that I prompted all of that. But the awareness that you have to use AI in conjunction with humans to validate software is where I'm still in agreement with where the future is going.

There was the big scare where people were being told — so why wouldn't they be scared? — that they were going to lose their jobs and QA was going to go away. So there was a big "whoa, get that away from me" from the QA industry. I think we've all caught our breath now. First of all, engineering is starting to settle down a little, because they're starting to say, "We can't go 10X. That's ridiculous. There's no way to get anything out at any quality at 10X." So the expectations are coming down a bit. At the same time, QA started to say, "They haven't fired us yet. Wait a minute — they need us more than they ever did." I think that's literally what we just saw happen.

In my space, I've seen a million and a half QA tools pop up. The agentic stuff is fantastic. I always sound like I'm anti-AI or anti-agentic, but I'm just holding the line: we have always used great tools, and we will always use great tools. This notion that we're working toward getting people out is wrong. It's about how you maximize the surface area you can cover with AI while retaining the really important pieces where human beings have to be for judgment. When you get that balance, maybe your devs are 4.5X — but they're really 4.5X. They're really getting stuff out, not some ethereal tech-bro version where we go as fast as we can and nobody cares about quality, because that's not real life.

I can see where that sentiment is coming from, and the tool market has reacted. To my point earlier about trust, the tools aren't trying to make you trust the platform in lieu of humans. They're trying to actively put guardrails around agentic workflows and equip QA people with the tools to stay in lockstep with developers as they gain all that velocity. That's what I've seen in this transition period, which matches what you're saying: there's a general sigh of relief in the QA community when it comes to AI, because now we know we can use it, and the general consensus is that we probably overreacted with the cuts and getting rid of the humans. So I think we're safe for now — whatever "now" is.

Ole Lensmar (06:40) Definitely. And even two or three X is a big improvement over one X.

Jay Aigner (06:45) You would have killed for 2X a year ago.

How do you trust tests that AI writes?

Ole Lensmar (06:49) Yeah. And to your point, the notion that verification is the bottleneck right now goes to so many different things. It's the mechanics of running tests and the coverage aspect, but also what I think is still the hardest part: how do you trust it? We just got asked this at another event last week. How do you trust the tests that AI is writing for you? How do you know it's not taking shortcuts or cheating, as we've all read that AI does in different scenarios? That's really still a hard part. My knee-jerk reaction is that you have another agent inspect what the first agent did and use AI to guardrail itself, which also feels like a meta problem. I still think there's a lot of experimentation that will happen. But similar to how quickly AI has been adopted for coding, I'm guessing it will go reasonably fast for testing as well. We'll see. Maybe, to your point, there are some borders people will not cross — for the helicopters and others — where there's a level they just don't trust. It'll take a while to figure out how to build harnesses or tools that give enough trust in those scenarios.

And the people you talk to at JDAQA — do they come to you with this problem specifically? How far along are they in general? It's always a bit of a challenge for us, because we're in the middle of the storm, so we're somewhat biased into thinking everyone's doing all this stuff. The reality check is often when we talk to prospects and find they're far away from where we think they are. What's your experience there?

Traditional, AI-enabled or agentic SDLC: where are teams today?

Jay Aigner (08:32) With most technology advances, it's a badge of honor to say you're doing the new thing. I agree totally. The number of teams I actually know that are really doing agentic the right way — not being pressured to deliver too much with no backing or reason to do it, and having the real balance — is small. I think there are still three buckets. There's the traditional SDLC. There's the AI-enabled SDLC, which is your Claude and prompt engineering, where people are still typing stuff in and doing code review. And then there's agentic. I don't think we get past agentic until we solve the trust problem. All my AI-forward CTO friends roll their eyes, but we're not going to get to the dark software factory soon. I'm not saying we won't, but I feel like we're settling into the buckets we're in right now.

Most of the companies I'm seeing are either in one of those buckets and establishing their footprint after moving to a new one — from traditional to AI-enabled, for example — or they're in a transition where they don't even know where they're at. That's why we've seen a lot of success with assessments since the spring: people just want to know where they are and what they should even be talking about. Recently, I had two CTOs on back-to-back calls tell me they hired a quality systems architect to solve the problems they heard us talking about at a conference in July. So new roles are being defined. I've called it a chief quality officer forever — the person who sits next to the CTO. When that person can say, "The stuff you're doing, you can't do it," and has a voice at the table, it's kind of the same as it's always been: you're building top-down process around the whole pipeline.

Even the most AI-forward people, who are very hesitant to say it out loud in public, are hitting the wall where they need those quality systems architects. They need someone to own it — someone to do all the stuff you just talked about, across all the different pieces along the way, because it's a new thing. That's why I generally think QA is dead: you can't just attack it at the end, or you'll be trying to backfill with people and tools forever and never really get there.

Beyond the last mile: what is spec recovery?

Ole Lensmar (10:41) Super interesting. Going back to how we trust what comes out at the end — is that the solution you see people going for, filling the gap with new roles? How are people attacking that? Is it exploratory testing at the end? What are some of the approaches you've seen to get past that last mile?

Jay Aigner (11:03) It feels like "the last mile" is still how everybody sees it, and I don't know if it's ever the last mile. We have to keep iterating, because we keep finding new places where we have to inject quality now that fewer humans are involved than before. There's nobody to point fingers at.

For example, we've started doing something I think is really cool, which we would have killed to do five years ago: spec recovery. As a testing person, you live and die by the spec. The software has to do what it's supposed to do. But you also know that in 90% of the organizations you come into, the specs are stale or wrong. So what we've been able to do is rebuild the documentation from a product perspective and then engage with the stakeholders to make sure it's correct. Then you have your bible, and you can build all of your AI guardrails based on this living spec concept that is continually version controlled and continually fed in from the system. As QA people, we now have this interesting ability to reverse-engineer the product and say, "We're going to make your system only able to do these things, so you'd better tell us if these are the right things, because this is what it's going to use" — whether it's agentic or you're using Claude and building a harness around it.

I think it's really cool that we can do things like that, which bring us up the value chain as QA people and QA agencies, because you couldn't do that before. For us to sit down and document any reasonably complex product used to be a big effort. So with things like that, we keep pulling ourselves forward and using the tools in new ways. I think there will be new roles, and new places in the pipeline where we'll have to keep finding the gaps that humans used to cover, now that we don't have fingers to point anymore.

Why the spec becomes the key artifact for AI

Ole Lensmar (12:46) That's super interesting, because it's one of those things QA has always been asking for: "I need a good spec so I know what to test. What are the functional and non-functional requirements?" Maybe now, with the advent of AI and the drive toward an AI-driven or agentic SDLC, everyone realizes we actually need a really good spec, because AI doesn't have all the inherent knowledge we humans have. Maybe that's actually a good thing that comes out of it, and the spec becomes a much more important artifact. Managing that spec, the context, and the inherent knowledge you have in an organization about your customers all needs to be captured so you can hand it off to agents. So maybe there's actually a good thing coming out of this that we haven't really had before. It's one of those things everyone wanted, but nobody took the time to create.

Being AI-forward, responsibly

Jay Aigner (13:43) Just so I'm super clear: I am super AI-forward. I come across a lot as if I'm negative on it, but this is the world we need to live in. We're here. I'm just saying, let's stop pretending and start clearing some of this stuff out. We have products that are built but unshippable. The only way to fix that is to build the AI infrastructure for the stuff we talked about, put people in the right places, figure out what your velocity settles at once you start doing that, and then incrementally increase it. I don't think it's rocket science.

We've got an opportunity as QA people, which is super exciting, to do all these things we've never been able to do. It almost highlights the fact that we never really realized why we needed these people before. We got it, but we really, really trusted them — and that's gone now, because those people aren't there. So some of the characteristics of the work we've done for the last 30 years are maybe highlighted more with AI. I am a million percent AI-forward. I just need to make sure the companies I'm working with do it responsibly and maximize the output while minimizing the risk.

Do testing tools still matter if AI picks them?

Ole Lensmar (14:50) Totally. Another thing I've been discussing with people, which is interesting to me: as we adopt AI more and more for testing and creating tests, the hypothesis is that the testing tools themselves become, not irrelevant, but not as interesting anymore. You'll just tell your AI, "I need an end-to-end test" or "I need a load test," and you may not be as concerned with which tool it chooses. Similarly, if you're vibe coding something for a hobby project today, maybe you don't have a strong opinion on which framework it uses. As long as it works in the end, you're happy. Does that make sense? Do you think the future is that we'll have functional and non-functional requirements at a high level and leave it up to the AI to decide on the tools? I'm sure there will be preferred tools that people always use, but testers might not really care whether it's Playwright, Cypress or whatever tool is coming down the line. Do you think AI creates an abstraction that makes the testing tool itself less important?

Jay Aigner (15:57) It's something we've talked about internally for a long time. I don't think we're there yet. The battle testing that software products have undergone is still very valuable to me as a QA agency owner who picks the tools we use.

I will say that was not my original thought. My COO, Autumn, always pulls me back to earth. I was like, "We're going to write all these test harnesses, do all this custom stuff, and roll it out to this client." And she said, "Jay, take a breath and realize we've used these tools for 10 years. We're not going to just switch over now because you can write something — and how do you even know what that is?" I'd taken the whole jug of Kool-Aid and pounded it. I thought it was going to be the greatest thing.

Do I eventually think it'll happen? At least Anthropic's and the AI companies' goal is basically to have no interfaces for most apps and just run the functionality through them. If you could just talk to Claude and do all the stuff you do with all the apps on your phone, why would you have the other apps? Claude would love that. So I think it's the same paradigm in testing, but I haven't seen it yet.

I will say I have seen people trust AI to write Playwright tests in a specific language and then do the code review that developers do. We do that: we have Playwright AI agents writing tests, and our people validate as it goes to make sure it's doing what it's supposed to do. I think the general public would eventually be fine with that today. Me as a QA guy, and I'd guess you too, would say it's probably not the right thing to do. So I'll ask the question back to you: why would somebody use a product — any product that's out there for QA — if they could just ask Claude?

Do testers still need to know Playwright?

Ole Lensmar (17:37) Maybe a different way of looking at it is that, as a tester, you won't need to know Playwright specifically. You might know there's a framework called Playwright that the AI uses to create end-to-end tests, but you don't need to be an expert in it. Having that abstraction AI provides may allow you to focus more on the spec and the high-level aspects of designing quality into your agentic pipelines, versus spending time learning the tool itself. I'm sure people will think, "What about selectors? There's so much complexity there." But I ultimately feel AI is at some point going to be good enough at creating tests with the frameworks we have that there will only be corner cases where people need to go in and manually improve those tests or tailor them to specific use cases. I'm probably going to get backlash for that one. Maybe. We don't know.

Jay Aigner (18:34) It's all tied to engineering, man. If they can get to the point where they trust it, then obviously we can. But I'm kind of tired of waiting and hoping the trust factor gets fixed. Let's deploy some software in the meantime, how about it?

Ole Lensmar (18:48) Okay, awesome. Thank you so much, Jay. This was super interesting. QA is dead or not — the discussion continues on LinkedIn and other channels. Thank you so much for joining, and thanks, everyone, for listening. The Cloud Native Testing Podcast is brought to you by Testkube. Have a great day. Goodbye.

Jay Aigner (19:03) Thanks, buddy.

Ole Lensmar (19:04) Thanks.