Investigating Our Own Past to Plan the Way Forward
October 16, 2025
Thursday morning at 8:26 AM, Code Agent deployed to investigate a test failure. Test 3 was returning HTTP 422 instead of success. Valid intent, proper authentication, should work — didn’t.
The natural conclusion: Phase 1 broke something.
The natural response: Roll back changes. Debug frantically. Fix “your” bug. Apologize for breaking tests.
Code Agent did none of these things.
Instead: 24 minutes of forensic investigation. Testing before the Phase 1 commit. Testing after the Phase 1 commit. Tracing the error back through git history to October 10.
At 8:51 AM, the finding: “Phase 1 changes are working correctly! The ServiceUnavailable error is PRE-EXISTING.”
This is the story of why systematic investigation prevents days of wasted effort — and how proving your work correct sometimes means finding what was broken before you arrived.
The test that failed
Test 3: Send a valid intent to the API. Expect success response. Get HTTP 422 instead.
HTTP 422 means validation error. Something about the request is malformed. But the test was sending a valid intent. Authentication was correct. The request format matched the API specification.
Everything should work. But didn’t.
Phase 1 had just been deployed. New error handling standards. REST-compliant status codes. The connection was obvious: Phase 1 broke the intent endpoint.
This is the moment where projects diverge. Roll back and debug? Or investigate systematically?
The choice: Investigation before assumption.
The forensic approach
Code Agent’s investigation sequence:
Step 1: Don’t assume Phase 1 broke it. Test the assumption.
Step 2: Check out commit 02ceaf06 (immediately before Phase 1).
Step 3: Run the same test against pre-Phase 1 code.
Result: Same error. But HTTP status 200, not 422.
This was the critical insight. The error existed before Phase 1. Phase 1 didn’t break anything — it exposed what was already broken by returning the correct HTTP status code.
Step 4: Trace the error back through git history.
Finding: ServiceRegistry gap from October 10 (commit d6b8aa09), five days earlier.
The problem: OrchestrationEngine depends on ServiceRegistry.get_llm() but the service wasn’t being registered in all startup paths. main.py registered services but didn’t start the server. web/app.py started the server but didn’t register services.
Phase 1 made this visible by converting the silent failure (HTTP 200 with error in body) into proper REST error (HTTP 422).
Investigation time: 24 minutes.
Days of wrong debugging prevented: Unknown, but likely multiple.
Proper fix enabled: DDD Service Container implementation addressing the root architectural gap.
What investigation revealed
The forensic work prevented wasted debugging time and also revealed architectural truth.
The root problem: Not in Phase 1’s error handling. In the service initialization pattern established five days earlier.
The proper solution: DDD Service Container pattern. Add LLM service initialization to web/app.py lifespan. Check if registered, initialize if needed. Enable independent server startup without breaking existing code.
Implementation time: 2 hours 50 minutes.
The payoff: Every subsequent phase ran 60–90% faster than estimated because the foundation was solid.
Phase 2 (15+ endpoints): 50 minutes versus 2+ hours estimated. 60% faster.
Phase 3 (test audit): 5 minutes versus 45–60 minutes estimated. 90% faster.
Phase 4 (documentation): 6 minutes versus 30–45 minutes estimated. 87% faster.
The time “lost” on investigation and proper fix paid exponential dividends in execution speed.
This is why investigation prevents waste. Not because it’s fast — because it’s correct.
The discipline of testing assumptions
The pattern that worked:
Don’t assume the recent work broke things. Test the assumption. Run the same test against pre-change code. Compare results. Let evidence guide conclusions.
Trace issues to root causes. When an error appears, find when it was introduced. Use git history. Test specific commits. Don’t fix symptoms without understanding origins.
Separate concerns clearly. Phase 1 was about error handling. The ServiceRegistry gap was about service initialization. These are different problems requiring different solutions.
Invest in proper fixes. The 2h 50min DDD Service Container implementation addressed the architectural gap completely. No workarounds. No “we’ll fix this later.” Proper solution enabling future velocity.
This isn’t just debugging methodology. It’s architectural discipline.
When something breaks, investigate systematically. When investigation reveals root causes, fix them properly. When proper fixes take time, invest it. The compound returns make the investment trivial.
Documentation bugs equal code bugs
Later that day, Phase Z validation caught something else: a critical documentation error.
The bug: Documentation examples showed {"intent": "show me standup"} but actual API expects {"message": "show me standup"}.
Impact: Would have confused all API consumers. Every example would fail. External developers would be frustrated. Documentation hotfix required. Credibility damaged.
How it was caught: Phase Z validation script ran real API calls, not theoretical examples.
This typo was a specification violation that would have broken all example code.
The philosophy: Treat documentation with the same rigor as production code. Documentation examples should be executable. Validation should run real API calls. Bugs in docs are bugs in the system.
The traditional approach: Write documentation, publish it, hope examples work.
The systematic approach: Documentation examples are code. Validate them in CI/CD. Catch errors before users see them.
One small field name mismatch. Massive downstream impact. Caught because we treated documentation like production code.
Testing reality versus testing ideals
Phase Z also revealed something about validation philosophy.
Initial approach: Validate against idealized REST behavior.
- Empty intent → 422 validation error
- Missing user → 404 not found
- Invalid workflow → 422 validation error
Actual behavior: System has intentional design choices.
- Empty intent → 500 (service layer validation, correct for this architecture)
- Missing user → 200 with defaults (intentional UX improvement)
- Invalid workflow → 404 (FastAPI routing)
Code Agent’s realization: “Test what works, not ideals. System works correctly; tests should validate reality.”
This is pragmatic quality: Test what the system does, not what textbooks say it should do. (I reviewed this with my Chief Architect to make sure we were not just “teaching to the test”.)
Intentional design choices aren’t bugs. Service-level validation has its place. Graceful degradation improves UX. Not every edge case needs endpoint-level validation.
Document these choices. Explain why they’re intentional. Don’t force conformance to textbook patterns when actual patterns serve users better.
The validation script evolved: Stop expecting idealized behavior. Start validating actual system behavior. Result: 5/5 tests passing with realistic expectations.
What Sprint A2 completion teaches
Wednesday completed Sprint A2. Five issues shipped. Zero regressions. 100% test pass rate.
But the remarkable thing wasn’t the metrics. It was the methodology validation.
Issue #142: Notion validation (78 minutes, proper investigation pattern)
Issue #136: Hardcoding removal (15 minute verification — already complete!)
Issue #165: Notion API upgrade (SDK + API version + data_source)
Issue#109: GitHub legacy deprecation (190 lines eliminated)
Issue #215: Error standardization (REST-compliant, validated, documented)
Every issue completed properly. No shortcuts. No “we’ll fix this later.” No technical debt accumulated.
The sprint demonstrated something important: When methodology emphasizes investigation over assumption, proper fixes over workarounds, and validation over hope — sprints complete successfully and sustainably.
The 24-minute investigation that started Wednesday wasn’t about saving time. It was about establishing truth. Phase 1 wasn’t broken — it was working correctly by revealing what was broken before.
The 2h 50min architectural fix wasn’t overhead. It was foundation that enabled 60–90% faster execution on all subsequent work.
The documentation validation wasn’t pedantic. It prevented every external developer from hitting broken examples.
The reality-based testing wasn’t compromising standards. It was documenting intentional design choices rather than forcing conformance to ideals.
The benefits of proper investigation
Looking back at the days’ work, the pattern is clear:
Early investigation (24 minutes) → Proper diagnosis (pre-existing issue) → Architectural fix (2h 50min) → Faster execution (60–90% throughout) → Sustained velocity (rest of sprint)
Without investigation: Days debugging wrong code → Wrong fix applied → Technical debt accumulated → Slower execution → Compounding problems
At least when it comes to working with forgetful AIs, the time spent investigating isn’t overhead. It’s the investment that prevents waste.
The discipline to test assumptions isn’t paranoia. It’s the practice that finds truth.
The commitment to proper fixes isn’t perfectionism. It’s the foundation that enables velocity.
Wednesday proved what systematic investigation enables: completing work correctly the first time, at sustainable pace, with quality maintained throughout.
What tomorrow will reveal
Sprint A2 completed. Sprint A3 about to start.
But today’s investigation pattern — systematic forensics, proper fixes, documentation rigor, reality-based validation — would prove even more valuable in the days ahead.
The methodology wasn’t lucky. It was systematic. The investigation discipline wasn’t overhead. It was foundation. The proper fixes weren’t perfectionism. They were quality that compounds.
Investigation prevents waste. Not by being fast, but by being correct.
When tests fail, investigate before assuming. When investigation reveals root causes, fix them properly. When proper fixes take time, invest it. The compound returns make the investment trivial.
A 24-minute investigation saved days of work. Not through speed — through discipline.