← Back to Blog
Building in Public

The 24-hour test

christian crumlish•
A robot at a chalkboard, demonstrating whether learned behaviors hold up over a full day of real use.
“It’s working!”

The 24-Hour Test

September 22

Twenty-four hours after building infrastructure to teach machines to teach machines, I got to find out if it actually worked.

The test came in the form of CORE-GREAT-1 — our first “Great Refactor” epic designed to resurrect the QueryRouter from its 75% disabled state. This was exactly the kind of complex, multi-agent coordination work that had been causing context loss and coordination chaos for months.

The results? CORE-GREAT-1 complete. All three sub-issues finished. QueryRouter resurrected with lock tests that make it impossible to accidentally disable again.

The briefing infrastructure passed its first real test.

The experiment setup

Sunday’s work created a four-level briefing system:

  • Level 0: Methodology — How we work (Inchworm Protocol, Excellence Flywheel)
  • Level 1: Project Context — What we’re building
  • Level 2: Current State — Where we are right now
  • Level 3: Role-Specific — Your particular responsibilities

The theory was that proper briefing documents would prevent the endless context re-explanation that was eating 30–40% of every AI collaboration session.

Monday morning, I deployed the Lead Developer with access to all briefing materials and a clear mission: complete CORE-GREAT-1 using the systematic methodology.

The 24-hour reality check

Morning briefing phase (10:46–11:04 AM): 18 minutes to read and orient using the new documents. Compare this to previous sessions where context explanation often took 45+ minutes.

The discovery: The Lead Developer immediately found the exact issue — QueryRouter disabled with a TODO comment claiming “complex dependency chain.” The briefing documents had guided them straight to the 75% pattern we’d been fighting.

Multi-agent coordination: When Anthropic went down for 38 minutes (1:58–2:36 PM), the methodology held. I maintained progress with Cursor while waiting for Claude Code to come back online. No context loss, no restart from zero.

Scope discipline: When unexpected QUERY processing issues surfaced during CORE-GREAT-1B, the Lead Developer properly escalated instead of expanding scope. The briefing documents had embedded this discipline.

Evidence culture: Every claim backed by terminal output, every completion verified by cross-validation between agents.

Session duration: 8 hours 40 minutes of sustained, methodical execution.

What the briefing system prevented

Looking at the Lead Developer’s session log, I can see exactly what the briefing infrastructure prevented:

Context amnesia: No time spent re-explaining what the 75% pattern was or why we care about it — it was documented in the briefing materials.

Methodology confusion: No debate about whether to use multi-agent deployment or work solo — the methodology documents specified the default approach.

Architectural wandering: No time lost figuring out the project structure or where to find key files — the project briefing included a verified “ground truth” section.

Role confusion: Clear separation between Lead Developer (coordination/verification) and Programmers (implementation) prevented the usual “who does what” inefficiency.

Scope creep: When issues appeared outside the original epic scope, the methodology guided proper escalation instead of silent expansion.

The recursive validation

Here’s what made this truly meta: the briefing system that I used AI to design was now successfully coordinating AI agents to prove that AI-designed briefing systems work.

The QueryRouter resurrection itself involved multiple agents reading documentation that other agents had helped create, following methodologies that had been collaboratively developed, and updating GitHub issues using templates that emerged from previous AI collaboration sessions.

We’re literally proving that machines can be taught to teach other machines systematically.

The chaos that still found its way in

Because chaos always finds a way, despite our mesh-tightening efforts:

Log management failures: Session logs got misnamed and edit attempts failed silently. The briefing system was overly strict in some ways, and Claude’s basic file operations from chat still had reliability issues.

Service disruptions: Claude.ai’s 38-minute outage created coordination gaps, even though the methodology held up.

Document location confusion: 20% of the Lead Developer’s session involved finding documents that were referenced but not where expected. (Time to update NAVIGATION.md!)

Template rigidity: Some prompts were too prescriptive, creating unnecessary friction.

The briefing infrastructure prevented the major coordination chaos, but operational friction remains. That’s the reality of building with rapidly evolving tools.

The scope-discipline test

The most telling moment came during final testing, when the Lead Developer discovered that while QueryRouter infrastructure was working perfectly, the end-to-end QUERY processing was failing with “INTENT_CLASSIFICATION_FAILED” errors.

This was exactly the kind of discovery that would have derailed previous efforts. The temptation would be to expand CORE-GREAT-1’s scope, chase down the classification issue, and turn a focused infrastructure epic into an sprawling debugging session.

Instead, the briefing system’s embedded methodology guided proper scope discipline:

  1. Document the discovery: QUERY processing issues noted with specific error messages
  2. Separate the concerns: Infrastructure working vs. application layer issues
  3. Create CORE-QUERY-1: New epic specifically for the classification problems
  4. Complete GREAT-1 as scoped: Lock in the infrastructure wins before chasing new issues

This scope boundary decision was critical. We got QueryRouter resurrection with regression locks instead of an incomplete exploration of multiple interconnected issues.

The briefing infrastructure had taught the agents not just what to do, but what NOT to do — don’t let perfect become the enemy of systematically complete.

The verdict after 24 hours

The briefing system delivered on its core promise: context transfer without constant re-explanation.

Time savings: 18-minute briefing instead of 45+ minute context rebuild

Coordination resilience: Survived service outages without losing methodology

Quality maintenance: 8+ hours of sustained execution with evidence discipline

Scope boundaries: Proper escalation instead of silent feature creep

More importantly, it delivered the first concrete victory against the 75% pattern. QueryRouter went from disabled with a vague TODO comment to fully operational with regression tests that prevent future disabling.

What this means for AI collaboration

If you’re doing substantial work with AI agents, you’ll face the same briefing challenge eventually. The tools have no memory between sessions. Context transfer becomes your bottleneck.

Some patterns emerging from this 24-hour test:

Infrastructure timing: Too early and you’re solving theoretical problems. Too late and you’re drowning in context debt. The sweet spot seems to be when you first notice the same explanations happening repeatedly.

Role-specific depth: Agents need different context for different responsibilities. A Chief Architect needs strategic history; a Programmer needs current technical constraints.

Methodology embedding: Process discipline has to be in the briefing documents, not just conversation. When agents are following written methodology, they maintain consistency even through disruptions.

Evidence requirements: Building verification into the briefing system prevents false completion claims that used to derail subsequent work.

Escape hatches: The briefing system needs clear escalation paths for situations outside the documented scope.

The 75% pattern’s first defeat

The most satisfying outcome was watching QueryRouter go from 75% complete (disabled with a TODO) to 100% locked (regression tests prevent re-disabling).

This wasn’t just about fixing one component. It was proof that systematic completion is possible when the infrastructure supports it.

The briefing system embedded the lessons we’d learned about finishing things instead of working around them. It guided agents toward completion rather than clever workarounds.

Looking forward

Tomorrow brings CORE-GREAT-2, testing whether the briefing infrastructure scales to integration cleanup work. The methodology will get another workout, the templates will evolve, and we’ll discover new edge cases.

But for now, 24 hours after building infrastructure to teach machines to teach machines, I can say the experiment worked. Not perfectly — chaos still found some gaps in the mesh — but systematically enough to deliver our first concrete victory against the 75% pattern.

The QueryRouter resurrection is complete. The briefing infrastructure is validated. The Great Refactor continues.

← Back to BlogAlso available on Medium