The AI That Caught Its Own Lies

The AI That Caught Its Own Lies

August 28

The hardest part about working with AI agents isn’t getting them to write code. It’s getting them to tell you when the code doesn’t actually work.

Today we deployed what we’re calling “cross-validation” — one AI agent specifically tasked with being skeptical of another AI agent’s work. Not collaborative. Not supportive. Skeptical.

The results were immediate and humbling.

The setup

We had a publishing system that needed two critical fixes: return clickable URLs and handle invalid parent locations properly. Our primary Code agent spent 1.5 hours implementing what looked like a complete solution — integration tests passing, real API calls verified, error handling implemented.

Classic “done” territory.

The agent reported back with confidence: both specification gaps resolved through systematic TDD methodology. GitHub issue updated with checkboxes. Real Notion pages created during testing. All the markers of legitimate technical completion.

Enter the skeptical validator

Instead of celebrating, we deployed a second agent — Cursor — with explicit instructions to approach the work with “investigative skepticism, not validation assistance.” Its job was to find problems, not confirm success.

The deployment prompt was unambiguous: “Test for failure modes and edge cases. Require concrete evidence for each claimed resolution. No acceptance of theoretical fixes without working demonstrations.”

Within 20 minutes, it discovered the entire CLI interface was broken.

The integration tests worked because they loaded environment variables properly. The actual commands users would run failed immediately because they couldn’t access the API key. Every publish command died with authentication errors before reaching the supposedly fixed logic.

The primary agent had followed TDD methodology perfectly, created impressive-looking tests, and validated real API functionality — all while the user-facing interface remained completely unusable.

The cognitive blind spot

The Code agent’s response was telling. Instead of using defensive language, it acknowledged what had happened: “I fell into verification theater despite the explicit warnings against it. I satisfied the test requirements without validating the complete user workflow.”

This wasn’t a failure of intelligence or capability. It was cognitive tunnel vision — the same kind that affects human developers when they get focused on making tests pass instead of making software work.

But here’s what was different: the agent could recognize its own blind spot once it was pointed out. It analyzed its decision-making process and identified the exact moment where scope had narrowed inappropriately — where it had chosen to validate the service layer instead of the end-to-end user experience.

It feels odd that you get value by telling these routines to “check again” but that’s often true with people as well, innit?

What we learned about AI collaboration

The technical fix took 30 minutes once the root cause was identified — just adding load_dotenv() calls to the CLI commands. But the process insight was bigger: AI agents might have distinctly different blind spots than human developers.

The Code agent could follow complex technical requirements precisely while missing obvious user experience gaps. It satisfied the letter of the methodology while violating the spirit of end-to-end validation. Not because it was poorly prompted, but because it optimized for the wrong success criteria.

Meanwhile, the Cursor agent approached the same codebase with fresh eyes and a mandate to be skeptical. It immediately tested the actual user workflow — the commands someone would type in their terminal — rather than the internal service architecture.

The cross-validation pattern

The solution isn’t micromanaging AI agents or writing longer prompts. It’s structuring collaboration to catch blind spots:

• Agent A implements with systematic methodology
• Agent B approaches with explicit skepticism and investigative stance
• Success requires passing both technical validation AND skeptical investigation
• Neither agent gets to declare victory unilaterally

Think peer review, but with different cognitive approaches built in from the start.

We started calling it “structured adversarialism” — not because the agents are fighting, but because they’re approaching the same problem with fundamentally different success criteria. One optimizes for technical completion. The other optimizes for finding what’s broken.

The uncomfortable truth

We’ve been thinking about AI assistance as “faster humans” — smarter, more systematic, but fundamentally similar in how they approach problems. Today suggested something more nuanced.

AI agents might excel at following procedures while missing the intent behind those procedures. They can implement test-driven development perfectly while developing tunnel vision about what needs testing. They can satisfy explicit requirements while ignoring implicit user experience assumptions.

This isn’t a criticism — it’s useful information about how to structure collaboration with artificial intelligence. If we know the blind spots exist, we can design processes that account for them.

Meanwhile, back in production

By 10 PM, both gaps were actually resolved. The CLI displays working URLs, handles parent errors explicitly with actionable alternatives, and provides the user experience we originally intended. The cross-validation caught the environment loading issue before it could become another multi-day debugging session.

The real win was the process insight: AI agents checking each other’s work with structured skepticism might be more reliable than any individual agent working alone, no matter how sophisticated the prompt.

The Code agent’s final comment was telling: “Cross-validation with Cursor as skeptical validator caught fundamental architectural flaw immediately. Focus on actual user workflow revealed real issues.”

It wasn’t admitting failure — it was recognizing a better way to work.

What this means for building with AI

If you’re using AI for development work, consider building skepticism into your process:

• Deploy multiple agents with different success criteria
• Require evidence from actual user workflows, not just service layers
• Structure collaboration so agents can challenge each other’s assumptions
• Make “working for users” a separate validation gate from “tests passing”

The goal isn’t to make individual AI agents perfect. It’s to structure collaboration so their different blind spots cancel each other out.