Today we launch Minitest, a fully autonomous AI QA engineer for mobile. This is a long argument about why we built it and what we believe we have changed in the way mobile QA gets done. If you lead engineering on a mobile team that ships every week and spends real money on tests that break for reasons nobody can explain, the rest of this is for you.

Every mobile team I have spoken to in the last 18 months can tell me about a release they cannot forget. CI was green and every smoke test passed, yet something still broke in production once real users got hold of it, usually a critical flow (onboarding, checkout, Adjust integration). The fix waited the standard App Store review cycle while the team wrote a postmortem, and the next release usually felt the same.

Calling that laziness misses what is actually happening, which is a slowly widening gap between what a test suite can verify and what a user actually does. The gap is older than mobile, agile, or test-driven development. In our view at minitap, this is the same gap that overtook chess engines a few years ago.

The bitter lesson keeps being relearned

In 2019, Rich Sutton, one of the people who built the foundations of reinforcement learning, wrote a now-canonical essay called The Bitter Lesson. His argument was that across every major domain in artificial intelligence research, the general methods that lean on computation eventually beat the methods that try to encode human knowledge, and the lesson keeps being bitter because the field keeps having to relearn it.

Chess is the canonical example. For about forty years, the strongest chess engines were the ones that encoded the most human chess knowledge: opening books and hand-tuned positional heuristics, with endgame tablebases bolted on. Stockfish, the strongest engine of the early 2010s, was built on that approach. In late 2017 DeepMind published AlphaZero, which had taught itself chess from self-play in roughly nine hours and won 28 of 100 games as White against Stockfish 8, drawing the rest, with no opening book or positional rules baked in. AlphaZero had a general method (self-play under deep reinforcement learning) paired with a great deal of compute, and the research consensus shifted in a single afternoon.

Human knowledge in chess was not wrong; it was a workaround for a general method that did not yet exist.

Software testing has had its own bitter lesson coming for fifty years

Software testing has its own version of this pattern. Each generation encoded a different kind of human knowledge into the suite, and each one's suite died for the same underlying reason: the encoded knowledge could not keep up with the product.

  • In 1970, Winston Royce's paper introduced what we now call waterfall, in which verification was the last phase before release. Tests were documents written by separate QA teams, and code shipped slowly enough that the lag was tolerable for the businesses that practiced it.
  • In the mid-1990s, Kent Beck inverted the order with test-driven development. The developer would write the test first, then write the code that made it pass, and the test itself became the developer's encoded knowledge of what the function ought to do. JUnit, which Beck and Erich Gamma published in 1997, made running unit tests cheap enough that the practice could spread.
  • In 2006, Dan North coined behavior-driven development with the ambition that the test should read as a plain-English specification a non-technical stakeholder could write. RSpec had shipped a year earlier and Cucumber followed in 2008, both built around the Given, When, Then form. The dream was that the test would describe the user's job rather than the user's clicks, and it did not survive contact with the implementation, because every Gherkin step still had to be wired up in code underneath.
  • In 2010, Jez Humble and David Farley's Continuous Delivery made tests the gate every change had to pass through. Mike Cohn's test pyramid, with many unit tests at the base and very few end-to-end tests at the top, became the budget under which suites had to live, because end-to-end tests were too expensive to maintain.

The dream of testing the user's job rather than the user's clicks was articulated almost twenty years before any agent existed that could deliver on it. That gap is what set the industry up for its bitter-lesson moment.

Mobile breaks before web does

The web has a Document Object Model, and the DOM gives selectors enough stability that record-and-replay can almost work. Mobile has nothing equivalent, which is why mobile teams feel a sharper version of the testing problem than their web counterparts ever did. A web hotfix ships in hours; a mobile fix has to clear App Store review, which can take days when something is urgent and longer when adoption lag matters across users on old binaries.

Most mobile teams I talk to settle for one of four workarounds:

  1. A small in-house E2E suite (Maestro, Cucumber, hand-rolled XCUITest) that dies within a quarter.
  2. BrowserStack or Sauce Labs plus an engineer who spends a meaningful slice of the week on manual QA.
  3. A manual-QA agency on retainer, billed by the hour against a coverage matrix the team rarely reads.
  4. Ship the build and hope nothing breaks.

If you lead a mobile team at a 3- to 30-person company, you have probably tried at least two of these and possibly all four.

Cursor, Claude Code, and the ever growing gap of untested code

When Cursor and Claude Code reached scale across 2024 and 2025, development velocity moved measurably faster on most feature work. Test authoring and maintenance stayed where they were when the Continuous Delivery book came out in 2010. The App Store hotfix window, painful but tolerable for years, became intolerable in the same eighteen months. The workaround stops working the moment one side of the equation accelerates and the other side does not.

We built the general method

In 2025 my co-founder Luc and our five-person team open-sourced mobile-use, a software development kit for agent-driven mobile UI interaction. We released it as a research experiment to find out how far an agent could be pushed on graphical user. We did not particularly expect the answer to make us a product company.

In 6 months, mobile-use reached 2.5k GitHub stars with just a few Reddit posts, and the top of Google DeepMind's AndroidWorld leaderboard among open-source agents, and we published the result. AndroidWorld measures the exact thing software tests have always tried to measure, and our agent does it directly rather than through steps or a script.

In Sutton's terms, AndroidWorld plus mobile-use was the moment in this domain where the general method beat the workarounds.

Testing the job behind the screen

Suppose your application has a login flow in which the user enters credentials and ends up on a home screen with a confirmation. The official Appium-WebdriverIO boilerplate, the canonical reference for Appium tests on native iOS and Android apps, carries a representative version of this kind of test:

it('should be able login successfully', async () => {
    await LoginScreen.tapOnLoginContainerButton();
    await LoginScreen.submitLoginForm({ username: 'test@webdriver.io', password: 'Test1234!' });
    await NativeAlert.waitForIsShown();
    await expect(await NativeAlert.text()).toContain('Success');
    await NativeAlert.tapOnButtonWithText('OK');
    await NativeAlert.waitForIsShown(false);
});

Behind those calls is a screen object referencing accessibility-ID selectors like ~button-LOGIN, ~input-email, and ~input-password (see appium-boilerplate, LoginScreen.ts). The test depends on three accessibility identifiers and an implicit timing assumption, any of which break the moment the team renames a button or restructures the form.

The Minitest equivalent of the same test looks more like this:

minitest
Ready to run

User flow

A logged-out user should be able to log in and reach the home screen

Acceptance criteria

  • Login completed
  • Home screen visible

The agent reads the application, runs the job a real user would run, and reports whether the job was completed. If a button moves on the next sprint the agent finds it, because the job is the same job.

One of our pilot customers was running A/B-tested onboarding experiments and renamed their primary call-to-action as part of the experiment. Their signup-completion suite kept running green throughout. In their product, the job stayed stable while the screen kept changing, so the job was the right thing to test.

This architecture has zero issues of maintenance and zero issues of flakiness, both of which sit on a script-based suite as technical debt. When a suite is brittle, someone has to own the fix and someone has to own the suite. Google's own engineering data put the cost of test flakiness at 2 to 16% of testing compute and roughly 1.5% of test runs (Google Testing Blog, 2016). The Minitap bet is that no engineer should ever have to carry that.

Why you would want an agent to run your suite

The first thing an agent gives you is agility. The maintenance bill ends. The senior mobile engineer stops spending Friday afternoons explaining to the team why a green build is actually red. Minitest is the only product built so the engineer does not own the suite, which means no one on your team has to keep it running.

The second is proximity to the user. We run the job a real user would run, on the build that is about to ship, against the same backend the production app uses. We minimise running in test environments and stay as close to production as is safe. We are fully autonomous: we build the app from source, so there is no time spent uploading builds or assembling a test rig. Parallel devices keep a full suite run fast, and broader device coverage is available when the test demands it.

The third is what the agent sees that no static test can. While the run is in progress, the agent is inspecting application logs in real time and watching CPU and memory usage the way an experienced engineer reading a profiler would. A script that taps Subscribe and asserts the home screen will not notice that the home screen rendered after a multi-second main-thread freeze. The agent does, because the agent is reading the same signals you would read if you were sitting next to the device.

The fourth is cross-platform maintenance by design. A script-based suite multiplies with every platform you ship: iOS and Android each demand their own selectors, their own flake, and their own maintenance bill, so n platforms cost n suites. Because the agent reads the running application instead of executing platform-specific scripts, one flow specification covers every platform without a parallel suite for each one.

That maps onto four categories of issue we surface:

  • Functional — the job does not complete, or completes incorrectly. Every script-based suite covers this.
  • Data and memory — leaks, retained allocations, CPU spikes, runaway main-thread work. The agent reads these directly from runtime logs while the flow is executing.
  • UI and UX — layout regressions, accessibility issues, text overflow, frame drops. The agent reads the rendered screen as a user would and surfaces these as it runs the flow.
  • AI feature — non-deterministic behaviour in the model-driven parts of your product. This category grows in importance every release as more of the product surface becomes model-driven.

Our internal headline metric is findings acted upon. A Minitest run only matters insofar as the team acts on what it surfaces, and every part of the product points at that single number.

A mobile app studio we work with runs Minitest across seven applications and fifteen releases per day with zero maintained tests on their side.

When a run goes red, the first failure shows up in Slack before the rest of the run completes, with a severity-first message of the form "Critical · Checkout — Apple Pay button unresponsive", a video with a timestamp, and a fix prompt the developer can paste straight into Cursor. When the run goes green, they ship in the evening and the on-call phone stays quiet.

Isn't this circular? AI testing AI code?

The most common objection from CTOs whose teams have already adopted AI coding assistants is that there is something circular about using an AI agent to verify code generated by another AI agent. The objection is reasonable, and the answer is structural.

Minitest validates user-facing behaviour independently of how the code was written. The test specification comes from the product behaviour the user is trying to perform, not from the implementation underneath. The agent runs a job a user would run and reports the outcome. Whether the engineer who wrote the underlying code was a senior staff developer or Cursor in agent mode is invisible to the test at run time. The bias people worry about, where humans write tests that conveniently pass for the code they wrote, is at least as old as TDD itself, and TDD has lived with that bias for twenty-five years. Architecturally, catching a regression does not require knowing who introduced it.

Three things to do this week

If you have read this far and are nodding, three actions are worth taking before the weekend.

  1. Stop investing in selector-based suites. The maintenance debt outpaces the team within about two quarters at the speed AI coding tools are accelerating development. Cap whatever you currently spend on that suite, and refuse to extend it.
  2. Pick on architecture rather than brand. Many of the agentic-QA tools that get press today still execute steps that someone defined in advance, whether those steps are described as flows, scenarios, or scripts. We derive jobs and run them. In 2027 the cohort that bet on human curation will look the way Stockfish-class engines looked in 2018: still useful, no longer the frontier.
  3. If you want to see what minitest's agent does, the evergreen demo at minitap.ai/demo shows it running end-to-end on one of our internal apps. When you want it on your own codebase, book a call and send us your App Store or Google Play Store link, and we will set up the first run on a flow you actually care about.

Mobile is first, and web and desktop sit on the roadmap

The architecture is not really mobile-specific. Once an agent owns a mobile test suite end-to-end, the same approach extends to web and desktop, both of which sit on our roadmap. The lane is QA. We start on mobile because the cost asymmetry is sharpest there, the in-house QA function is usually missing, and the App Store penalty made the math break first.

That's a wrap on mobile QA

Stripped to its core, the argument is this: the agent owns the suite, you ship, and it tests, which means the team carries none of the authoring, maintenance, or curation burden a script-based suite would impose. What you want, when we talk about this, is some version of a green light on your release. You want to know that Subscribe still works after the redesign, that the integration with Stripe did not silently break when their SDK shipped a new version yesterday, and that the thing you are shipping behaves the way you think it does. We answer that question, and we measure ourselves on a single number: whether teams act on what we find.