AI & ML
Your AI builder shipped the Stripe code in an afternoon. Verifying it took a week.
FetchSandbox DEV Community
2 views
A friend shipped a booking app last month. Lovable built the whole thing, Stripe Checkout included, in something like an afternoon. It looked finished. It was finished, in the sense that you could pay and get a confirmation email.
Then a customer got charged twice for one booking.
Not a double-click. One payment, one payment_intent.succeeded, delivered twice, because Stripe retries. The handler ran twice and created two bookings. Nobody caught it in review, and here is the uncomfortable part: the code was fine on the happy path, and nobody reviewing it had written a line of it.
That is the actual shape of the problem right now. Build time collapsed. Verification time did not.
The generated code is not the problem
This is where I expected to find sloppy AI output, and I didn't. The handler read exactly like something a competent developer would write on a first pass:
app.post('/webhook', async (req, res) => {
const event = stripe.webhooks.constructEvent(req.body, sig, secret);
if (event.type === 'payment_intent.succeeded') {
await createBooking(event.data.object.metadata.booking_id);
await sendConfirmation(event.data.object.receipt_email);
}
res.json({ received: true });
});
Signature verified. Correct event type. Returns 200. It will pass code review and it will pass a smoke test, because the first delivery does exactly what it should.
There is no idempotency key anywhere in it. The agent was never asked "what happens on the second delivery," so it never answered that question. Neither did the reviewer, because you cannot see a retry in a diff.
Why the usual tools miss it
Mocks return the response you configured. A mock does not decide to retry you. That is the whole point of a mock, and it is also why it cannot catch this class of bug.
Staging proves the happy path against real credentials. To see a duplicate delivery you have to wait for Stripe to genuinely retry, which means causing a timeout on purpose and waiting out a backoff. Nobody does this on a Tuesday.
stripe trigger fires a fresh, well-formed event. Useful, but every trigger is a first delivery. The bug lives in the second one.
So the failure mode is invisible to all three, and it surfaces when real money moves.
Force the second delivery
What I actually wanted was a Stripe that behaves like Stripe, including the annoying parts, on demand. That is what a service twin is: stateful, so a created PaymentIntent can be read back later, and scriptable, so you can ask for the retry instead of waiting for it.
FetchSandbox runs as an MCP server, so the agent that wrote the integration can drive it. In Cursor or Claude:
Run the Stripe accept_payment workflow, then re-run it under the
webhook_retries scenario. Tell me how many bookings exist afterwards
and give me the receipt URL.
webhook_retries delivers each webhook once, then replays it twice with the same upstream event id at backoff intervals, which is the behaviour that matters. A handler deduping on a per-delivery header still fails here, because that header changes on every retry while event.id does not.
The run came back with three bookings for one payment. Not an opinion about the code, a count.
The fix is the boring one:
if (await alreadyProcessed(event.id)) {
return res.json({ received: true });
}
await recordProcessed(event.id);
Then re-run the same scenario and the count stays at one. That buggy-to-fixed flip is the thing worth having, because "the agent says it fixed it" and "the retry no longer double-books" are different claims.
The part that makes it reviewable
Every run produces a receipt at a shareable URL: the requests in order, the state the twin held, the webhook events including the replays, and a verdict. The verdict is separate from the HTTP status on purpose, so a run can return 200 at every step and still come back unproven.
That distinction is what makes this useful when you didn't write the code. You are no longer reviewing an unfamiliar diff and hoping. You are reading evidence that a specific failure was forced and handled. Put the URL in the PR.
The same run works from the CLI when you want the pipeline to enforce it rather than a person remembering:
fetchsandbox run <sandbox-id> accept_payment \
--scenario webhook_retries --json
Local proof while building, the same check gating merges. The failure branch becomes a required step instead of a thing you meant to test.
Beyond Stripe
Payments are where it bites hardest, but the pattern is identical anywhere an integration has a lifecycle. Paddle, Resend, Twilio, Clerk, Descope, and AgentMail have curated workflows and failure scenarios; the wider catalogue covers 50+ APIs generated from their OpenAPI specs. Same loop each time: run the flow, force the failure, keep the receipt.
Setup is one MCP config block, and there is more on the approach for AI-built apps if you want the longer version.
The actual question
I keep meeting people whose apps went from idea to paying customers in a weekend, and whose integration testing story is "staging looked fine."
So, genuinely: what are you doing here? Is anyone verifying the retry, the dropped webhook, and the mid-flow auth failure before production, or is the honest answer that you find out from a support ticket? I am more interested in what people actually do than what we all agree we should do.
Read original: https://dev.to/fetchsandbox/your-ai-builder-shipped-the-stripe-code-in-an-afternoon-verifying-it-took-a-week-2o6n
← Previous
Common AWS Free Tier Mistakes Beginners Make
Next →
OpenBSD Stories: Strange Medieval Devices
Related
How to Turn Any Photo Into a Cross Stitch Pattern
AI & ML
0
Dev.to (EN Zone)
I built an AI website builder for Webround. Then I killed it.
AI & ML
0
Dev.to (EN Zone)
Supercharging DeepSeek Harness: Bringing Claude Pro/Max with One-Click Google/Gmail OAuth Login and Real-Time Quota Tracking
AI & ML
0
Dev.to (EN Zone)
A free image-generation API with no key and no account — and the three failures that look like success
AI & ML
0
Dev.to (EN Zone)
Comments0
No comments yet — be the first