Quick Summary
This article walks through a real vibe coding testing workflow we built for a client’s AI-generated application two weeks before their launch date. We cover how we structured the testing layers, what the 40+ bugs actually were, how specific prompts created specific failures, and what repeatable system we left behind so the team could keep shipping without shipping blind.
Table of Contents
Most vibe-coded apps get demoed. Very few get tested.
There is a difference, and it matters more than most founders realize until something breaks in front of a real user. A demo exercises the happy path: the button works, the data is saved, and the screen loads. Vibe coding testing exercises everything else: what happens with the wrong input, a dropped connection, a user who does something the prompt never imagined.
The gap between those two things is where bugs live, and in AI-generated codebases, that gap is wider than most people expect. A 2025 analysis by Cursor’s engineering team found that AI-generated code introduces edge case failures at roughly twice the rate of human-written code, not because the AI writes bad logic, but because it writes exactly what you asked for and nothing more. It has no instinct for what could go wrong. It has never seen a production incident.
This article is about building a system that catches what the AI does not think to prevent.
It came in on a Tuesday. A founder, two weeks out from launch, had realized their AI-built app had never been tested by anyone who was actually trying to break it.
The product was a project management tool built for small creative agencies: time tracking, client reporting, invoice generation, built on a React frontend and Supabase backend. Built almost entirely through prompts in Cursor over eight weeks. The UI was good. The core flows worked. Their version of testing had been the founder clicking through the app after each prompt session and confirming screens loaded.
That is not testing. That is a demo with extra steps. When we asked what their QA process looked like:
They came to us because a co-founder had asked a question nobody could answer: what happens when one client tries to access another client’s invoices? Nobody had tried it. Nobody knew.
The honest reason they came to us instead of running more prompts: another round of prompts would have added features. It would not have found the holes already there. The AI builds forward. Finding what is broken requires someone looking backward.
Standard QA assumes the code was written by someone who understood the full system. A human developer who builds an authentication flow has usually thought about what happens if the token is missing, expired, or belongs to someone else. They may not have handled it perfectly, but they have thought about it.
AI does not work that way. The model satisfies the prompt in front of it. It is not holding the rest of the system in mind. It is not asking what could go wrong three screens later. And when a new prompt rewrites a component, it has no memory of why the previous version was written the way it was.
That last point is what makes vibe coding testing categorically different from standard QA:
The regression problem is the hard one. An early prompt adds an authorization check. A later prompt, asking for something totally unrelated, regenerates that component and drops the check because nobody told it the check needed to stay. The screen looks identical. The bug is invisible until someone exploits it.
This is also why teams building with vibe coding tools like Cursor, Lovable, or Bolt need a testing layer that runs after every meaningful prompt session, not just before launch. By the time you are two weeks out, you are not catching bugs early. You are hoping you catch them all.
The other thing standard QA misses is prompt-induced scope creep. When a founder steers an AI through eight weeks of feature building, the codebase reflects every pivot, every change of mind, and every “actually, make it do this instead.” Human QA checks that features work. It rarely checks whether the features that got replaced left something half-connected behind. QA for vibe coded apps has to account for that explicitly, or it is checking the wrong things.
We scoped the engagement as four passes, run in a specific order. The order matters because each layer catches a different class of failure and informs the next one.
| Layer | When It Runs | What It Catches |
|---|---|---|
| Prompt-Level Rules | Before generation | Missing auth, exposed secrets, no input validation |
| Static Analysis | After each generation | Injection patterns, unsafe handling, dead code |
| Behavioral Testing | After features are wired | Edge cases, wrong tokens, cross-user data access |
| Regression Pass | After every prompt cycle | Safe behavior silently removed by a later prompt |
Before a single line of new code was generated, we put a security and quality rules file in the project root. Every prompt the team ran would inherit those standing instructions: enforce access checks, validate input before using it, never put credentials in the frontend, keep authentication logic intact when refactoring.
It does not stop every problem. But it eliminates whole categories of mistakes before they appear in the codebase, which is the cheapest possible place to stop them. On this project, the rules file alone prevented three classes of issues we had seen on previous engagements from appearing at all.
After each generation session, we ran Semgrep and ESLint across the changed files. Automated tools are not enough on their own, but they are fast and consistent, and they catch patterns a human reviewer will miss after the fourth hour of reading code.
What they found on this project: two injection-adjacent patterns in edge functions, one unused dependency that had been pulled in by a prompt and never removed, and a handful of console logs that were printing request data in production. None of these were critical on their own. All of them were the kind of thing that compounds when combined with a larger exploit.
This is where the real bugs lived. We mapped every route the app exposed, including the ones the UI never linked to directly, and sent requests the interface would never send:
Twenty-three of the 40+ bugs came from this layer alone. None of them would have shown up in a demo. Several of them would have shown up on day one of real usage, when a real user did something the founder had never thought to try.
We ran this after every significant round of prompting, not just at the end. The rule was simple: if a prompt touched a file that had previously passed testing, that file got re-tested from scratch.
This caught eleven bugs that had been introduced by prompts that had nothing to do with security or data handling. A performance optimization. A UI reskin. A copy change that somehow triggered a component rewrite. In each case, something safe in the previous version was gone in the new one, and nothing in the output signaled that anything had changed except the thing that was asked for.
Our vibe coding cleanup specialists at Bacancy audit AI-generated codebases, build the testing layer your prompts never added, and re-test everything properly before real users hit it.
The full log had 43 issues. Here is how they broke down by category:
Three examples worth tracing back to their prompts:
Prompt: “Build the invoice dashboard so it loads all invoices for the logged-in user.”
What the AI did: Filtered by user ID at the UI level, not the query level.
What that meant: Changing the user ID in the request returned another client’s invoices. The screen would never let you do that. The endpoint did not care.
Prompt: “Make the time tracker persist across navigation.”
What the AI did: Wired persistence to local state, which survived navigation but not a refresh.
What that meant: Any user who refreshed mid-session lost their tracked time silently. No error, no warning, just gone. On a time tracking product, that is not a minor UX issue. That is the core feature failing in a way that would have produced support tickets from day one.
Prompt: “Clean up the reporting page layout.”
What the AI did: Rewrote the component cleanly. Did not carry over the role check from the previous version.
What that meant: Any logged-in user could access report data scoped to admin roles. The page looked the same. The gate was gone. This one would not have been obvious in testing unless you specifically tried to access the page as a non-admin, which standard QA would have had no reason to do.
This is the pattern that repeats across every AI-generated codebase we review. The bugs are not random. Each one is a logical response to a prompt that did not mention what needed to be preserved. This is also a key reason why teams using vibe coding for MVP development need a structured testing layer from the first sprint, not a patch job before the last one. The longer regressions sit in a codebase, the more subsequent prompts build on top of broken foundations.
The vibe coding security matters here too. Several of the authorization gaps we found were not just quality issues. They were exploitable. A competitor, a disgruntled user, or a moderately technical bad actor could have used them. The line between a bug and a vulnerability in AI-generated code is thinner than most founders expect, and it is one of the reasons a proper audit catches things a standard QA pass never would.
Fixing 43 bugs before launch was the immediate job. Leaving behind a system the team could run themselves was the one that mattered longer term. If you hire vibe coding developers who only patch what is broken today, you are back in the same place after the next sprint.
We left three things:
This is the piece most teams skip entirely, and it is the most important one to get right. QA for vibe coded apps is not the same as standard feature testing. Standard testing confirms that features work. This pass confirms that the safe behavior from the previous version survived the latest prompt. It runs after every session that touched existing files, not just new ones, and it specifically tests for the things AI tends to drop quietly: auth checks, access rules, input validation, and session handling.
The checklist for this pass is short by design. Long checklists do not get run. This one fits on a single page and takes under ten minutes for a typical prompt session output.
Short, practical, built around the failure patterns we had just documented. The main rule: any prompt that touches a file with existing auth or data logic needs to explicitly say “keep the existing access controls intact.” The AI will not assume. You have to tell it every time, because it has no memory of why the previous version was written the way it was.
The secondary rule: any prompt that asks for a refactor or cleanup needs to be followed immediately by a behavioral test of the file it touched. Cleanup prompts are the single most common source of regression bugs in AI-generated codebases. They look harmless. They are not.
The founder’s team now treats every prompt session the way a team with a human developer treats a pull request: something that works is not the same as something that is ready to ship. That single shift in mindset does more for long-term product quality than any individual fix we shipped.
The app launched on time. Same product, same eight-week build advantage, minus the 43 issues that would have hit real users in the first week. The founder’s instinct to ask before launching was the decision that made the difference.
Vibe coding testing is not the thing that slows you down. It is the thing that makes the speed worth keeping. The economics of building with AI only hold if what ships actually works when someone who is not the founder starts using it. A fast build that fails in production is not a fast build. It is a slow recovery with a deadline attached. The workflow we built on this project is not complicated. Four layers, a checklist that runs in minutes, and a prompt discipline that takes thirty seconds to apply. What it requires is the decision to treat AI output as a first draft that needs review, not a finished product that needs deployment.
If you are building this way and want it done right, look for vibe coding services that include a structured testing layer from the first sprint, not a cleanup pass before the last one. The bugs are cheaper to catch early. They always are, and with AI-generated code, there are always more of them than the demo suggests.