I was standing in a hallway at Droidcon USA in Orlando a few days ago, talking to developers from Cash App, Marshmallow, and Tinder. Different companies, different products, different stacks, and yet we were all describing the same thing. All of our companies are pushing dramatically more code with LLMs, and all of us are scrambling to deal with what comes after. The folks at Block have open-sourced Trailblaze, an AI-driven test recorder that drives tests on Android and iOS. I told them about the framework I'm building at DoorDash. And someone said the quiet part out loud: "We need to figure out a way to drive these tests automatically, because human QA is almost becoming impossible at the scale we're building and shipping features."
Then I walked over to the sponsor booths and had the same conversation again. Two startups there, Maestro and Drizz, are building in exactly this space. Chatting with the founders, we ran through the growing list of others: Mobile Boost, Momentic, AutoSana. Everybody is trying to use AI to run these tests, and everybody is running into the same wall: when an AI agent drives a test, it leans on a multimodal model or a view hierarchy dump at every single step, and the tests get painfully slow. How do you make verification fast and deterministic?
By the third conversation with a developer from Tinder, the pattern was undeniable. Everyone is quietly working on the same problem: how do we verify what our apps actually do, now that we can no longer read all the code?
How trust used to work
Think about the safety net we used to have. At most companies I've worked at, code baked in master for one to two weeks before a release. A pair of human eyes went over every PR. Unless everyone was rubber-stamping, at least someone glanced at your code. Static analysis ran its checks. You tested your own work before merging. Some teams even hosted testing parties or bug bashes before a every release.
That chain also leaked, of course. Missed by review, missed by QA, straight into production. Having QA never meant you'd never ship a bug.
But looking back, the old system felt safer for a simple reason: everyone wrote code at roughly the same pace. Unless someone was truly exceptional, their output wasn't more than double anyone else's, and on average, everybody produced an amount of work that could reasonably be tested within a sprint. The whole system worked because code entered it at human speed.
The dam broke
That's over. The amount of output a single person can generate now varies wildly, and it varies because of AI.
I'm not writing about "those people." I'm one of them. Right now, almost 100% of the code I work on is generated by AI. I read maybe half of it, mostly to learn what I'm doing. For an internal tool that only a few developers will ever see, I'll often read very little; if I break it, I patch it in minutes. When I was building my testing framework, essentially an enhanced fork of Maestro, under time pressure and working solo in a separate repository, at some point I stopped reading the code entirely. I verified behavior, PR after PR. As long as the behaviors I expected existed, I merged.
And "just review it more carefully" doesn't scale as an answer. It only works if everyone on your team can actually catch things in review, but you can generate so much slop that people simply can't follow it. I remember developers who blocked off their early hours just to review code. Now, even when you leave feedback, the author's AI responds to it and acts on it. The narrative shifts to: why do I bother reviewing somebody's code when they themselves won't read my feedback, only their AI will? The motivation to review properly quietly collapses.
Even Meta has accepted this. They've built a pipeline that assesses the risk of a PR, using not just AI but heuristics, decision trees, and machine learning, to decide which PRs can be auto-stamped and which genuinely need human attention. The results they share are remarkable, and I'll break that paper down properly in a separate post. I might even try implementing it.
The confession
Here's the uncomfortable part, and I'll own my share of it.
I always worked well with QA engineers, arguably too well. I leaned on them. I shipped things half-hoping that if something was wrong, QA would catch it before it reached production. What I personally hated was writing Espresso tests, or any UI tests, really. They were slow, and I told myself that as long as my code was well isolated, unit tests were enough. Best architecture, single responsibility, everything testable in isolation: if the unit tests pass, the code is fine.
In practice, it almost never works that way. You're usually working in a legacy product, your new code interacts with old code, and the only thing that tells you the truth is integration or behavior testing, what some people call end-to-end. Your unit tests pass and your code is, more often than not, still not fine.
And as an industry, we did this collectively. Most engineers never picked up the testing tools at all, and now we're paying the price: shrugging, being wishy-washy, "I don't know how to write tests," "I've never written tests," "that's a QA engineer's job." I've always believed the engineer who wrote the feature has the responsibility, ownership, and accountability to also test its behavior. They know best what cases it's supposed to handle. We outsourced that discipline for decades, and now it's about to become the core skill of the job, because verification, not generation, is the scarce resource.
The moment it really hit me was this past quarter, knee-deep in building the testing framework at DoorDash. It's trivially easy for an AI to write unit tests that pass on the code it just wrote. If you think "my unit tests passed, so the code must be right," you're living in a delusion of productivity. But if the agent can prove to you, visually, through a video recording, that the behavior you expected is actually happening, that gives me real confidence in what my agent produced. People need to learn to have their agents produce verification results, not just code: an adversarial agent or, an evaluation agent running alongside the builder. It's not about having the best model or the biggest context window. It's about demanding proof that what you asked for is what you received.
Why the tools aren't enough
The frameworks have existed for decades: XCTest, Espresso, Appium. The honest truth is they don't fall short so much as we never learned them. The new AI-native wave promises that agents will automatically verify everything in your app, but they're all slow, all doing essentially the same thing, and none of them can give you real confidence in a reasonably short time. If a single test takes three to four minutes on average, you can't run that suite on every pull request. Even a nightly regression suite becomes a slog, or you shrink it until it barely covers anything. I've felt the rough edges of these tools personally; my Maestro Unicode workaround exists because Maestro couldn't input Korean text due to an ADB limitation, and I had to build an entire alternate-keyboard, tap-driven input path just to type. The gap is real. The deep technical evaluation is for future posts.
Where I sit
This is the problem I'm actively working on at DoorDash. I'm building a framework that looks like Maestro, with your tests in YAML, except it's built to run really fast and deterministically, with no AI in the runtime loop. People use AI to author the tests; the execution stays predictable. I'm thinking hard about parallelism: running tests across many emulators on a beefy bare-metal machine, so we can verify far more in far less time, with a suite big enough to give confidence at the PR level, not just nightly.
The Droidcon conversations gave me conviction that this matters. When the most senior people in mobile independently converge on the same problem, that's the signal. Because AI agents are pattern-matching machines: if there are bad patterns in your codebase, they'll compound them. If you can't verify what you're shipping every week or two, quality degrades, and users notice. In an era where anyone can spin up a product on Replit or Lovable in a weekend, every SaaS is an incumbent under siege. The engineering moat is setting the quality bar so high that no proof of concept can touch it.
An invitation, not a conclusion
I don't have the answer yet, and I think saying so is the honest move. The last twenty years were about learning to write software faster; the next few are about learning to trust it again.
What I genuinely don't know: where does AI belong inside the tests? I get that you can use it for assertions and self-healing, but I can't accept the lag that comes with it, unless there's a way to run a cheaper, faster model on the host, or lean on embeddings, without losing robustness. Today I only use AI to author tests and to diagnose failures: was it flaky, was it written wrong, was it a real bug? I want my framework to become truly AI-native, and I haven't figured out what that means yet.
So if you're an engineer, tech lead, or engineering manager feeling the pain of an ever-growing codebase, if productivity went up but quality went down, if your SLAs keep getting breached: I want to talk to you. Let's share notes, brainstorm, maybe collaborate, and figure out how to get back to pre-AI levels of quality without driving ourselves into the ground. Reply, or DM me @droidchef.
