Digging Out of the Complexity Hole

June 17–18, 2025

“I want us digging back out of this hole, not deeper into it.”

That was my plea to Claude halfway through what had started as a simple multi-project support implementation and had somehow devolved into a 30-hour debugging marathon involving duplicate domain models, broken import chains, and the kind of architectural confusion that makes you question everything you thought you knew about your own codebase.

This is the story of what happens when complexity compounds faster than your ability to manage it, and how systematic recovery sometimes requires admitting you’ve been digging in the wrong direction.

When simple features aren’t simple

PM-009 was supposed to be straightforward: add multi-project support so users could work with different repositories without manually specifying which one every time. We had patterns established, domain models in place, a test suite written. How hard could it be?

Famous last words.

The first sign something was wrong came when our migration script couldn’t figure out which Project class to import. Turns out we'd accidentally created two completely separate model hierarchies: services.domain.models.Project and services.database.models.Project.

Both were called Project. Both served similar purposes. Neither knew the other existed.

The architectural awakening moment

“Glad I checked. It is already not a good ‘scent’ that we are chasing bugs on this script. It makes me wonder where we were not careful enough to begin with.”

That was my moment of recognizing we weren’t dealing with simple bugs — we were dealing with architectural sloppiness that had been accumulating without anyone noticing.

Sometimes the most productive thing you can do is stop trying to make broken architecture work and start figuring out why it’s broken in the first place.

When test-driven development meets reality

Here’s what made the situation particularly frustrating: we’d been following test-driven development principles religiously for the earlier tickets. Write tests first, implement to pass the tests, refactor carefully. It had worked beautifully through PM-008.

But PM-009 was different. Complex business logic, multi-layer coordination, multiple decision paths. And somewhere in that complexity, we’d abandoned TDD discipline without realizing it.

I should have instructed by assistants to read the test specifications carefully and implemented exactly what the tests expected, but instead they guessed at method names and signatures, and then we spent hours debugging why our llm.infer_project_id() calls didn't match the llm.complete() methods the tests were mocking

The tests weren’t wrong. Our implementation was wrong. But we’d been so focused on making the code work that we’d stopped paying attention to what the tests were actually telling us.

The duplicate models anti-pattern

The root cause of our architectural confusion was hiding in plain sight: we’d created two separate Project classes because an earlier example had shown the structure but not the naming conventions.

# What we accidentally built:
services.domain.models.Project      # Pure business logic
services.database.models.Project    # SQLAlchemy persistence

# What we should have built:
services.domain.models.Project      # Pure business logic  
services.database.models.ProjectDB  # SQLAlchemy persistence

Seems obvious in retrospect, right? But when you’re deep in implementation mode, small naming decisions can have huge architectural consequences.

By the way, usually when I write “we realized” or “we discovered” here, what I really mean is that this was happening in collaboration with Claude as my architectural thinking partner. But the moment of stepping back and saying “wait, why do we have two Project classes?” — that came from forcing myself to slow down and actually look at what we’d built instead of continuing to patch symptoms.

The cost of violating TDD discipline

The most sobering discovery was realizing how far we’d drifted from test-driven development without noticing.

Six test failures, all from method signature mismatches. Six opportunities to recognize we weren’t following our own process. And we’d been interpreting each failure as “something’s wrong with the environment” instead of “something’s wrong with our approach.”

Complex features don’t need less discipline — they need more discipline. But complexity has a way of making you feel like you don’t have time for proper process.

The systematic recovery approach

Once we recognized we were digging deeper instead of climbing out, we took a systematic approach to recovery:

Step 1: Stop patching symptoms No more “quick fixes” to make individual tests pass. No more environment tweaks to work around import issues.

Step 2: Fix the root architectural problem Renamed Project → ProjectDB in the database layer, added explicit to_domain() and from_domain() mapping methods.

Step 3: Follow the tests exactly Changed method calls to match what the tests were expecting, not what we thought made sense.

Step 4: Verify the foundation Made sure basic imports and database connections worked before attempting business logic.

The recovery wasn’t glamorous. It was mostly mechanical refactoring and careful attention to details we’d been glossing over. But it worked.

An architectural lesson about examples

One insight from the recovery was how architectural examples can mislead if they’re not complete.

We’d been following an example that showed the structure of domain and database models but didn’t specify the naming conventions. So we created two Project classes instead of what we eventually eneded up with, a domain class called Project and database class called ProjectDB.

Good examples don’t just show structure — they show naming, relationships, and boundaries. Incomplete examples can be more dangerous than no examples at all.

Ancknowledging complexity

I’ve already made it clear that I find things complex enough as they are and strive to avoid adding any optional complexity, but there comes a time when you need to recognize that some things worth making are by necessity complex.

Once we were on an even keel. I commented “OK it makes sense that things have gotten harder and more complex. (whew)”

That was my moment of accepting that PM-009 represented a genuine complexity jump from the earlier tickets. Not just more code, but more kinds of complexity:

Business logic complexity: Multiple decision paths for project resolution
Integration complexity: Coordinating between domain models, repositories, and LLM calls
State management complexity: Session-based project memory across user interactions

Complexity isn’t a failure — it’s a natural progression as systems grow. But complexity requires acknowledging its presence and adapting your process accordingly.

When architectural refactoring pays off immediately

The moment we fixed the duplicate models issue, everything started working. Not “sort of working” or “working with workarounds” — actually working the way it was supposed to.

# After the refactor, imports became obvious:
from services.domain.models import Project        # Business logic
from services.database.models import ProjectDB    # Persistence

# Mapping became explicit:
project = project_db.to_domain()                  # Clean conversion
project_db = ProjectDB.from_domain(project)       # Clear direction

This is what clean architecture gets you: when you fix the foundation, everything else becomes easier.

The test environment archaeology

Part of our debugging odyssey involved figuring out why our test environment had become unstable. The discovery process was like archaeology:

Layer 1: pytest-asyncio version incompatibility
Layer 2: Import path inconsistencies with shared_types
Layer 3: NumPy 2.0 compatibility issues in ChromaDB dependencies

(Oh, NumPy compatibility issues, you numpty! I just can’t quit you.)

Each layer revealed assumptions we’d made about the environment that were no longer true. Each fix revealed the next layer of issues.

Environment issues can mask architectural problems, making you think the problem is with your setup when it’s actually with your code.

The multi-AI coordination reality

One surprising aspect of this stage of development was that discovery that even with multiple AI assistants providing review and guidance, we could still all drift into architectural anti-patterns together.

Claude Opus: Architecture review and pattern recognition
Claude Sonnet: Detailed implementation planning
Cursor Copilot: Code execution and syntax assistance

Multiple perspectives, multiple checks, and we still ended up with duplicate domain models.

AI collaboration doesn’t eliminate the need for architectural discipline — it just distributes it across more participants.

The business-logic debugging phase

Once we had clean architecture, we could focus on the actual business-logic bugs:

Inference vs. last-used logic: When LLM inference conflicts with session memory
Default project confirmation: Whether users need to confirm obvious defaults
Ambiguous project handling: How to handle “UNCLEAR” responses from LLM
Missing project errors: Proper error types for non-existent projects

These were the bugs we should have been debugging from the start. But architectural confusion had made it impossible to isolate business logic issues from infrastructure issues.

Clean architecture creates clean debugging.

The 10/16 recovery milestone

“10/16 tests passing (major improvement from 2/16)”

That was our recovery milestone. Not perfect, but systematic progress in the right direction. From nearly complete failure to clear success, with specific remaining issues that had clear solutions.

Sometimes the most important measure isn’t how many tests are passing — it’s whether you’re making systematic progress or continuing to accumulate problems.

Discovering documentation debt

The architectural refactoring revealed that our documentation was describing a system that didn’t match what we’d actually built. Domain models that weren’t properly separated, import patterns that had evolved beyond recognition, test strategies that assumed simpler architecture.

We’d made a big effort at documentation, used that to power through the first nine github issues, and along the way forgot to update the docs to capture ongoing decisions and changes. Now they were pretty stale.

This is how technical debt compounds: inconsistent code leads to inconsistent documentation, which leads to incorrect assumptions in future development.

Good documentation isn’t just helpful — it’s a forcing function for architectural clarity.

A lesson about stopping versus pushing through

The most important lesson from the complexity hole was about knowing when to stop pushing forward and start digging back.

Signs you’re pushing forward: Making progress on intended features, tests passing, clear next steps

Signs you’re digging deeper : Fighting the same issues repeatedly, accumulating workarounds, architectural confusion

The temptation when you’re deep in a complex implementation is to keep pushing until you break through. But sometimes the breakthrough requires changing direction entirely.

The tension between systematic and tactical

Throughout the session, there was constant tension between systematic approaches and tactical fixes. Cursor would unfailingly recommend or (even just go ahead and start making!) quick fixes to get the tests to pass, and I had to keep telling it to stop as systematically dug into the root causes.

Tactical approaches feel faster in the moment. Systematic approaches are faster over time. But when you’re in the middle of complexity, it’s hard to trust that systematic work will pay off.

The recovery proved that systematic approaches work — they just require faith during the implementation.

Architectural discipline pays off

By the end of the 30-hour session, we had:

Clean separation between domain and database models
Explicit mapping patterns that prevented future confusion
Test environment that worked reliably
Business logic bugs that were isolated and fixable

None of this was glamorous work. Most of it was cleaning up mistakes we’d made earlier in the session. But it created the foundation for everything that came after.

This recovery work enabled the systematic implementation successes that (spoiler!) were to follow in the next weeks.

Managing complexity

What I learned from digging out of the complexity hole is that complexity management isn’t about avoiding complexity — it’s about managing it systematically.

Good complexity management: Acknowledging complexity, adapting process, maintaining discipline

Bad complexity management: Pretending complexity doesn’t exist, abandoning process, accumulating technical debt

Complex features require more discipline, not less. But when you’re drowning in complexity, discipline feels like a luxury you can’t afford.

When recovery becomes learning

The most unexpected outcome of the complexity hole was how much we learned about our own system architecture. The debugging process forced us to understand relationships and dependencies we’d been taking for granted.

Sometimes you don’t really understand your architecture until you break it and have to fix it systematically.

Learning to dig out of these complexity holes systematically has enabled us to recognize and address technical debt before it becomes overwhelming.