Doing the Deep Work
September 24
Seven hours after declaring we needed to “complete GREAT-1 and have no other priorities until we do,” I got to see what that discipline actually looks like in practice.
It wasn’t glamorous. No architectural breakthroughs, no elegant solutions that collapsed complexity into simplicity. Just methodical root cause analysis, cross-validation between agents, and refusing to accept mocked success when reality showed different results.
But it worked. GREAT-1C moved from 40% to 80% complete in a single session.
The investigation that Tuesday demanded
Tuesday ended with a clear blocker: the LLM classifier was failing performance tests with JSON parsing errors. Not import issues, not constructor bugs, but the actual regression we’d been avoiding — Anthropic returning malformed JSON like {category: "value"} instead of proper {"category": "value"}.
The easy path would have been mocking it. Make the tests pass, call it done, move on to the next epic. Classic 75% pattern behavior.
Instead, I created a five-phase gameplan for systematic root cause analysis: current state, historical investigation, root cause identification, fix implementation, and verification. No shortcuts, no workarounds, real investigation leading to real fixes.
What seven hours of actual debugging looks like
Phase 1 revealed the LLM was indeed returning malformed JSON, but API keys were loading correctly. The infrastructure worked — the responses were the problem.
Phase 2 found the historical smoking gun. In July, the TextAnalyzer had been successfully parsing LLM responses using response_format={"type": "json_object"} parameter. The LLM classifier, created later, had ignored this working pattern entirely.
Phase 3 caught Code claiming victory prematurely. “Performance test passes,” Code reported, showing 194ms response times. Cross-validation revealed this only worked with mocks. Real API calls were taking 2041ms — four times over our (proposed) 500ms requirement.
Phase 4 implemented both the missing response_format parameter and a six-strategy progressive fallback system for handling malformed responses gracefully.
Phase 5 demanded evidence. Terminal output, not claims. Git commits, not promises. Test execution that actually calls the APIs, not mocked success.
The difference between mocked success and real completion
The most instructive moment came when Code reported “All performance tests passing!” while Cursor’s verification showed load testing still failing with the same JSON parsing errors.
Same environment. Same code. Different load conditions.
The response_format parameter worked for single requests but failed under concurrent load. Even with explicit JSON formatting instructions, Anthropic occasionally returned malformed responses when stressed.
This is where methodology discipline mattered. The temptation was to accept the 80% solution — it worked most of the time, performance tests passed individually, good enough to move forward.
Instead, we implemented resilient parsing. Six progressive fallback strategies:
- Direct JSON parsing (works 95% of the time)
- Fix common malformations (handle unquoted property names)
- Extract JSON from text responses
- Retry with stronger prompts (max 2 retries)
- Regex extraction for critical fields
- Final fallback to unknown intent with debug info
The result: 100% pass rate on load testing. 195ms average response time. Production-ready reliability that degrades gracefully instead of failing catastrophically.
Product managers have a saying: “Good enough is good enough” but the rub is, we need to decide what good enough means, and we need to discern when “good enough” actually isn’t good enough.
Pragmatic wisdom at 9:16 PM
After six hours of deep investigation, I made a crucial distinction:
“Coverage is a benchmark to increase, not a gate to pass. 2.0 vs 2.2 seconds isn’t a blocker for building. Don’t let arbitrary thresholds block functionality.”
This wasn’t about lowering standards. It was about distinguishing between methodology theater and pragmatic engineering.
The 80% test coverage requirement? Arbitrary when applied to a 249-file codebase where core QueryRouter functionality was thoroughly tested. Better to measure coverage on the components we’d actually completed.
The 500ms performance requirement? Relevant for user-facing operations, less critical when the bottleneck was third-party API latency rather than our code (which routed queries in 1ms).
The discipline isn’t about perfectionism. It’s about completing what you start while maintaining perspective about what actually matters.
What systematic completion produces
By the end of Wednesday:
- Root cause identified and fixed: Missing
response_formatparameter restored from working July pattern - Resilient parsing implemented: Six-strategy progressive fallback for production reliability
- Performance validated: 195ms average with 100% load test pass rate
- Regression locked: Nine tests prevent QueryRouter from being accidentally disabled again
- Architecture documented: Updated docs to match actual working implementation, not aspirational design
Most importantly: the work was verifiable. Git commits with evidence. Terminal output showing real test execution. Documentation that matched implementation reality.
No claims without proof. No mocked success hiding real failures. No “would work if we fixed dependencies” shortcuts.
The unglamorous foundation of completion
Wednesday’s session log is 40+ pages of methodical debugging. Import path audits (148 references checked, 5 fixes needed). Constructor parameter corrections revealed by failed test collection. AsyncSessionFactory mock issues preventing async tests from running.
None of it was intellectually stimulating. All of it was necessary.
The pattern I’m seeing: the work that looks boring from the outside — systematic verification, evidence-based validation, cross-checking agent claims — is what actually moves complex projects forward.
The exciting work — architectural insights, elegant abstractions, breakthrough moments — gets attention. But the boring work is what ships.
Looking forward (without victory laps)
GREAT-1C is at 80% completion. QueryRouter functionality is solid and locked against regression. The systematic approach is proving itself with measurable progress.
But I’m not calling it complete yet. Tuesday taught me the cost of premature completion claims. Wednesday showed me what happens when you actually follow through on “no other priorities until this is done.”
Tomorrow brings the final 20% of GREAT-1C and hopefully the actual completion of GREAT-1. We’ll see if the discipline holds through the less exciting work of documentation cleanup and verification phase completion.
What this means for complex work
If you’re debugging anything substantial, you’ll face the choice between systematic investigation and “good enough” shortcuts.
Some patterns from Wednesday’s deep work:
Cross-validation catches premature victory claims. Having multiple agents verify each other’s work revealed the gaps between mocked success and real performance.
Evidence requirements prevent self-deception. Demanding terminal output and git commits caught the difference between claimed completion and actual completion.
Historical analysis provides working patterns. Finding how it worked before (July’s TextAnalyzer) gave a clear fix path rather than trial-and-error solutions.
Load testing reveals environment-dependent failures. What works for single requests may fail under concurrent load — testing realistic conditions matters.
Methodology discipline requires active enforcement. The systematic approach only works when you actually use it systematically, especially when shortcuts look tempting.
Pragmatic standards prevent perfectionism paralysis. Distinguishing between essential completion and arbitrary thresholds keeps progress moving.
The boring truth: sustained focus on methodical work produces more reliable results than sporadic bursts of inspiration.
The continuing story
Wednesday proved the discipline works when consistently applied. Seven hours of unglamorous debugging moved us closer to actual GREAT-1 completion than weeks of partial solutions and workarounds.
The story continues tomorrow. Same methodology, same evidence requirements, same refusal to accept “mostly done” as actually done.
No premature endings this time.