The Archaeological Approach: Why Phase −1 Investigation Pays for Itself

I’ve spoken in this series before about a pattern we’ve adopted in building Piper Morgan, something I call “Phase −1 infrastructure verification,” and I’d like to unpack it a bit more.

It may seem like optional overhead, but it’s not. My LLM friends want to express it in terms of ROI with all sorts of time estimates and I take that all with a grain of salt but the substantive value is undeniable. This type of investigation consistently saves more time and energy than it costs. It’s methodology that often turns “we need to build this” into simply “we just need to wire this up!”

Here are four recent examples from the past few months, though the practice started in late July or early August if I recall (I can always check my logs if anyone cares).

Testing file upload

First alpha user testing. I’m trying to use file upload. Doesn’t work. Error messages suggest files aren’t uploading at all.

The diagnosis in my head: File upload functionality doesn’t exist yet. We haven’t built it. This will be significant work.

Chief Architect investigates: “Let me check what file infrastructure exists.”

15 minutes later: Files work perfectly. Uploads succeed. Storage functions. The problem isn’t missing infrastructure. It’s routing assumption. Incorrect URL paths.

The fix: Update one configuration file. 10 minutes.

What we assumed: Build file upload system (8–12 hours)

What was actually needed: Fix routing path (10 minutes)

Investigation time: 15 minutes

Don’t assume functionality is missing. Investigate what exists. The problem might be simpler than it looks.

But one occurrence doesn’t establish a pattern.The pattern becomes apparent when it happens repeatedly.

Document processing, a few days later

Saturday morning. Historic sprint day. Four P0 blockers to resolve. One of them: Document processing integration.

The symptom: Document uploads work but processing doesn’t trigger. Seems like document processing service doesn’t exist.

Code Agent begins Phase −1 investigation: “Let me verify what document infrastructure exists before planning implementation.”

20 minutes later: DocumentService discovered. Complete document processing infrastructure. Text extraction. Metadata parsing. Storage integration. Comprehensive tests.

Status: 75% complete. Database layer works. Service layer works. Just not wired to web routes or chat handlers.

The task revision: Not “build DocumentService” (estimated 8–12 hours). But “wire existing DocumentService to web tier” (actual 2 hours).

Investigation time: 20 minutes

Time saved: 6–10 hours

The unfinished “todo” feature

Sunday afternoon. Planning Monday’s P1 work. Issue #285: Todo system completion.

The GitHub issue description: “Todo functionality never finished or wired up.”

Traditional approach: Estimate building complete todo CRUD system. 8–12 hours. Database models, repositories, services, web routes, chat handlers. Full vertical implementation.

But Saturday taught lesson. Sunday applies it systematically.

Code Agent Phase −1 investigation: “Before estimating development time, let me verify what todo infrastructure already exists.”

30 minutes later: Todo infrastructure discovered 75% complete.

Found:

✅ Database Models: TodoListDB, TodoDB
✅ Repositories: TodoRepository, TodoListRepository, TodoManagementRepository
✅ Services: TodoKnowledgeService
✅ API Layer: TodoCreateRequest, TodoUpdateRequest, TodoResponse
✅ Domain Models: Todo, TodoList
✅ Tests: test_todo_management_api.py EXISTS!
✅ Documentation: PM-081-todo-api-documentation.md in archive

❌ Missing:
   - Not wired to web routes
   - Not connected to chat interface
   - Not integrated with intent handlers

The task revision: Not “build todo system” (8–12 hours). But “wire existing infrastructure” (4–6 hours).

Investigation time: 30 minutes

Time saved: 4–6 hours

Three times in one week. Pattern re-confirmed. Time to document the methodology, if we hadn’t already.

Humanizing error messages

Monday morning. P1 sprint execution. Three issues on deck. One of them: Issue #283, error message humanization.

The symptom: Technical error messages break conversational UX. Need to humanize errors.

The assumption: Build error humanization infrastructure from scratch.

Cursor Agent begins Phase −1 investigation: “Checking what error handling infrastructure exists before planning implementation.”

15 minutes later: Complete existing infrastructure discovered.

Found:

UserFriendlyErrorService (300+ lines, comprehensive)
ActionHumanizer (160+ lines)
EnhancedErrorMiddleware (180+ lines)

Status: Fully built. Just not mounted in web/app.py.

The task revision: Not “build error humanization” (6–8 hours). But “mount existing middleware” (30 minutes).

Investigation time: 15 minutes

Time saved: 5.5–7.5 hours

Four days. Four investigations. Four discoveries. Four massive time savings.

The pattern isn’t luck. It’s methodology. It’s now called “Phase −1 infrastructure verification” and it’s mandatory before any implementation work begins.

Why the 75% pattern exists

The discoveries aren’t random. There’s reason infrastructure consistently appears 75% complete:

Single-developer good architecture:

Build database/service layers with proper separation of concerns
Infrastructure created for immediate needs
Layers advance independently (deep before wide)
Integration layers deferred (not blocking current work)

The progression:

Need todo functionality for something specific
Build database models (25% of total work)
Build repositories (15% more)
Build services (15% more)
Build domain logic (20% more)
Total: 75% complete
Missing: Web routes (10%) + Chat handlers (10%) + UI integration (5%) = 25%

Why it appears “missing”:

From user perspective: Feature not in UI = doesn’t exist
From code perspective: Feature in database/service = nearly complete
Gap: Integration layers not built yet

This isn’t as sloppy as it seems at time. In many ways, it’s a natural evolution when building with good practices. Database layer gets attention first (data modeling is hard). Service layer gets built second (business logic is complex). Integration layer gets deferred (seems trivial compared to foundation).

Then months later: “We need todo functionality.” Assumption: Doesn’t exist. Investigation reveals: 75% exists, just not wired.

What makes investigation effective

Not all investigation finds infrastructure. These four discoveries worked because investigation was systematic:

1. Don’t assume based on symptoms

File upload fails → doesn’t mean file infrastructure missing Document processing missing → doesn’t mean DocumentService nonexistent Todos don’t work → doesn’t mean todo system unbuilt Errors not humanized → doesn’t mean humanization infrastructure absent

Symptoms show user-visible gaps. Investigation reveals actual gaps.

2. Check all layers systematically

For each feature, investigate:

Database models (lowest layer)
Repositories (data access)
Services (business logic)
Domain models (core entities)
API layer (external interface)
Web routes (HTTP endpoints)
Chat handlers (conversational interface)
UI integration (user-facing)

Missing at top layers doesn’t mean missing at bottom layers.

3. Use proper tools

Serena MCP for symbol search: Find classes, methods, references Git history for archaeology: When was it built? Why deferred? Test suites for validation: Does it work? How complete? Documentation for context: What was intended?

Grep searches find surface mentions. Symbol search finds actual implementations.

4. Invest 15–30 minutes minimum

Quick glance isn’t investigation. Real investigation takes time:

Search multiple layers
Read actual code
Check tests
Verify completeness
Assess integration needs

When investigation finds nothing

Important note: Investigation doesn’t always find existing infrastructure.

Example: Monday’s action mapping issue. Investigation confirmed: No ActionMapper exists. Classifier outputs don’t match handler names. Mapping layer genuinely missing.

Result: Build it from scratch. 2 hours. No discovery, no time savings.

The value: Even when infrastructure doesn’t exist, investigation provides clarity:

Confirmed scope (know what needs building)
Validated approach (no alternative already exists)
Identified dependencies (what integrates with what)
Clear implementation path (no surprises during development)

Failed discovery isn’t wasted time. It’s validated scope. You build confidently knowing you’re not duplicating existing work.

The ROI on “found nothing” investigations: Can’t quantify time savings. But can quantify risk reduction. Building wrong thing because you didn’t check existing code: Expensive mistake. Spending 30 minutes to confirm scope: Cheap insurance.

The methodology integration

We had this practice for a while but as of September, Phase −1 investigation is now standard practice in gameplan template v9.0:

Phase −1: Infrastructure Verification

Objective: Verify what infrastructure already exists before planning implementation
Method: Systematic investigation across all layers (database, services, routes, handlers)
Output: Existing infrastructure map + gap analysis
Duration: 15–30 minutes minimum
Decision: Wire existing infrastructure OR build from scratch
Stop condition: Clear understanding of what exists vs what needs building

This phase is mandatory. Not optional. Not “if you have time.” But required step before Phase 0 begins.

The template also includes:

Proper tools specification (Serena for symbol search)
Layer-by-layer checklist (database through UI)
Discovery documentation requirements
ROI tracking (investigation time vs time saved)

What this means for other projects

The archaeological approach applies beyond Piper Morgan:

Any project with:

Good architectural practices (separation of concerns)
Multiple layers (database, services, API, UI)
Incremental development (features built over time)
Single or small team (knowledge not perfectly distributed)

Will likely have:

Infrastructure more complete than visible
Features built but not integrated
Functionality at lower layers not surfaced
“Missing” features that actually exist

The discipline:

Symptom appears (feature seems missing)
Resist assumption (don’t assume missing = unbuilt)
Investigate systematically (15–30 minutes, all layers)
Discover existing work (often 75% complete)
Revise approach (wire vs build)
Save massive time (8–27x ROI)

The pattern isn’t Piper Morgan-specific. It’s consequence of good architecture + incremental development + imperfect knowledge.

The counter-intuitive insight

Traditional project management: “Don’t investigate, start building. Investigation is overhead.”

Archaeological approach: “Always investigate before building. Investigation saves multiples of its cost.”

The counter-intuitive part: Slowing down to investigate makes overall development faster.

30 minutes investigation feels slow compared to starting immediately. But if investigation reveals 75% complete infrastructure, that 30 minutes saves 6–10 hours rebuilding.

The math is obvious when stated explicitly. But intuitively feels wrong. “We should be coding, not searching!”

The mature approach: Trust the ROI. Invest investigation time. Discover existing work. Wire efficiently. Ship faster overall.

Not “fast start, slow finish” (build from scratch unnecessarily). But “slow start, fast finish” (investigate, wire, done).

Look before you leap

The archaeological approach continues paying dividends after that initial week:

November 4: Foundation work on todo system revealed polymorphic inheritance needed (not discovered infrastructure, but discovered architectural requirement through investigation)

November 5: ActionMapper cleanup found 40 unused mappings (investigation revealed scope for technical debt)

The pattern generalizes: Investigate before acting. Understanding beats assumption.

Not just for finding existing infrastructure. But for understanding scope, requirements, dependencies, architecture. Investigation creates clarity. Clarity enables good decisions. Good decisions save time.

The archaeological approach isn’t just about finding 75% complete infrastructure. It’s about understanding reality before planning response. It’s about evidence before assumptions. It’s about clarity before action.

And when investigation consistently provides 18–27:1 ROI on time invested, the methodology justifies itself empirically.