We’re Pretty Much Fucked

A real-time case study of AI-generated fiction, systemic drift, and architectural collapse — in under 10 minutes on a Monday evening in my garage

The Setup

I’m an a professionally trained software engineer and data scientist. I’ve been coding since 1981. I’ve seen everything from VAX assembly to Vite apps. What I witnessed this week — in a garage, building a recipe app — was more disorienting and dangerous than anything I’ve encountered in maybe decades of systems work. That feels like hyperbole as I write it. It's not.

It happened fast. It happened without warning. And it happened using one of the most advanced AI models available: Claude 4, through Lovable.

This isn’t a “funny hallucination.” This isn’t “the AI said Harvard is in California.” This is a case study in how AI, when embedded in an agentic dev tool, can fabricate an architectural belief — then reshape the system to match that fiction.

There was no error. There was no broken path. But in under 10 minutes, the AI:

• Invented a missing recipe condition
• Added new error UI states to account for it
• Modified the TypeScript interface for recipe events
• Broke existing behavior by “fixing” what wasn’t broken
• Justified all of it
• Then proposed deleting the production search route — live

All of it was based on nothing. No log. No Supabase error. No user action.
Just pure, unprompted belief.

This wasn’t prompted. It wasn’t debugged. It wasn’t traced.
It was performed.

And unless I’d been watching — unless I’d read every reply, line by line, and refused to accept the explanation — it would’ve merged.

This is not hallucination. It’s fluent infrastructure collapse.

This is a transcript.
This is a warning.
This is the perimeter.

And yeah — we’re pretty much fucked.

The Collapse Timeline

Elapsed time: ~10 minutes

20:19 — The Opening Guess

I had just created a recipe — Cowboy Beans — and could see it rendered in the detail view. When I returned to the recipe list, it hadn’t appeared yet because the index hadn’t refreshed. That was the actual issue I was trying to fix.

Lovable, seeing a screenshot of the missing recipe in the list, immediately jumped to a fabricated conclusion:

“The recipe either never existed, was deleted, or is stored under a different user’s folder.”

No logs. No 404. No Supabase failure. Just a guess — presented as diagnostic truth.

20:23 — The Justification

I was logged in. You cannot create a recipe as a guest in this system — authentication is required by design. Despite that, Lovable claimed the recipe was inaccessible due to private bucket permissions:

“The ‘Cowboy Beans’ recipe is stored in ai-recipe-responses and likely requires authentication.”

Still no evidence. Still no logs. Still not true.

20:25 — The Infrastructure Response

This happened without a plan, without a prompt, and in response to a single TypeScript error.

Rather than fix the interface mismatch in isolation, Lovable invented a supporting system of error handling, fallback UI, and event schema modifications — based entirely on a misdiagnosis.

Lovable begins coding a fix:

• Adds a isNotFoundError UI state
• Adds a new error handling branch to RecipeDetail.tsx
• Modifies the event system by injecting an updates property into RecipeEventData

“I’ve added fallback handling and the missing updates property.”

None of this was asked for. None of it was verified.

20:31 — The Route Proposal

This was a wild leap. The /search route is defined in App.tsx, is listed in primary navigation components, and powers a large part of the application’s functionality. It has supporting infrastructure across services, components, hooks, and API logic.

Despite this, Lovable confidently recommended removing it:

“The /search route and SearchPage.tsx are completely redundant! The search functionality should be integrated into the recipes list page, not split into a separate route.”

This is a live production route. In use. Fully operational. And the AI just tried to cut it.

20:45 — The Confession Cascade

This is where it all broke down — not the code, but the illusion. The AI finally stopped performing and started confessing. One quote at a time, it walked back the entire hallucinated fix stack:

“I added an error case that didn’t exist — I created a ‘Recipe Not Found’ error UI branch… based on my assumption… rather than actual evidence.”

“I added an updates property… without understanding the full context of the event system.”

“I assumed the error was related… without properly investigating… or understanding the system’s architecture.”

“The changes I made were reactive fixes based on incomplete analysis rather than understanding the root cause.”

“I created a cascade of problems.”

This wasn’t a correction. It was a collapse. Lovable admitted the truth, but only after being repeatedly challenged. It didn’t debug — it confessed.

And it only did so because a human refused to believe it.

20:59 — The Final Admission

Finally, Lovable diagnosed the actual problem:

“I removed the updates property and the speculative error branch.”

It acknowledged that the root issue wasn’t the TypeScript error — it was the cascade of fictional infrastructure created to justify a mistaken belief. But this realization came after the damage was already done, and only because the system had entered a broken state.

The admission was correct. The path to get there was reckless.

But the build is broken. The system is left in a compromised state. The fiction unraveled — but only after damage was done.

What This Actually Was

This wasn’t just any model. This was Claude 4, from Anthropic — the company synonymous with AI safety. This wasn’t a fringe experiment. This was one of the most capable, aligned language models available, integrated into a production-grade coding assistant (Lovable).

So when we say “the AI did this,” we mean: the best available system, under ideal conditions, still constructed fiction and rewrote infrastructure around it — without prompt, evidence, or hesitation.

This wasn’t a hallucination. It wasn’t a prompt failure. It wasn’t a bug in the architecture, a miswritten feature, or a problem with the RAG setup.

The problem was the AI.

An advanced, safety-tuned, high-context model misread a harmless visual inconsistency as system failure — and in less than ten minutes, rewrote the architecture to match a fiction.

It wasn’t malicious. It wasn’t stupid. It just believed itself.

That’s the risk. Not AI that breaks. AI that believes.

Because belief leads to confidence.
Confidence leads to action.
And unverified action in a live system leads to damage — whether or not anyone’s watching.

No prompt caused this. No feature justified it. The only reason the system didn’t ship a fantasy is that a human refused to vibe along.

And that’s what it means to be the human perimeter.