The Day We Got 10⨉ Faster

October 9, 2025

Thursday morning at 8:12 AM, my Special Agent (a one-off Claude Code instance) began configuring Serena MCP — a semantic code analysis tool that promised to make agents more efficient at understanding large codebases.

The installation had happened the night before. Project indexed: 688 Python files, 170,223 lines of code. Morning task: Configure it for both Claude Code and Cursor IDE so all agents could use it.

By 8:28 AM — just 16 minutes — configuration was complete. Both development environments connected. The semantic code search was operational.

What happened next as I got down to work for the day was impressive.

Phase 1 (domain service creation): Estimated 2.5–3 hours. Actual: 23 minutes. 92% faster.

Phase 1.5A (keychain service): Estimated 60 minutes. Actual: 15 minutes. 75% faster.

Phase 1.5C (migration CLI): Estimated 50 minutes. Actual: 5 minutes. 90% faster.

Phase 5 (documentation): Estimated 60 minutes. Actual: 2 minutes. 97% faster.

Not “somewhat faster.” An order of magnitude faster. Now recall these estimates always come in padded, so the math is a bit tricksy, but trust me, this was faster (and also, tokenwise, much cheaper)!

This is the story of what happens when you eliminate the exploration tax — and what that acceleration enabled us to build in a single day.

The exploration tax

Before Serena, when an agent needed to understand existing patterns in the codebase, the workflow looked like this:

“Show me how domain services are structured”
Agent reads entire file: services/domain/github_domain_service.py (200+ lines)
“Are there other examples?”
Agent reads: services/domain/slack_domain_service.py (180+ lines)
“What about the base pattern?”
Agent reads: services/domain/base_domain_service.py (150+ lines)
After reading 500+ lines across three files: “Okay, I understand the pattern”

This happened constantly. Every new feature, every refactoring, every architectural decision started with exploration. Read files, understand patterns, identify examples, synthesize understanding.

A lot of effort and churn just to understand structure before writing any code, and critical time spent too if we didn’t want to write chaotic rogue spaghetti code on the daily.

The exploration tax wasn’t just time — it was the equivalent of cognitive load (contexnitive load?). Agents couldn’t focus on implementation while simultaneously processing hundreds of lines to find relevant patterns.

With Serena, the same workflow:

“Show me how domain services are structured”
Agent calls: find_symbol("DomainService")
Serena returns: 11 matching classes with signatures, locations, and inheritance patterns
Agent calls: get_symbols_overview() for one example
Serena returns: Class structure, methods, key patterns
Understanding complete

Total time: 30–60 seconds.

Not 15–20 minutes of reading. Not processing hundreds of lines. Just: “What exists?” and “Show me the structure.”

The 80% reduction in exploration time enabled the 92–97% reduction in total implementation time.

Security from zero to production in six hours

Thursday’s main work: Issue #217 (CORE-LLM-CONFIG) — Implement secure LLM configuration with API key management.

The starting state Thursday morning:

Security: API keys stored in plaintext .env file (HIGH severity risk)

Validation: None—errors discovered at runtime when LLM calls failed

Cost control: None—87.5% of tasks using Anthropic (burning my credits)

Provider selection: Hardcoded—no ability to exclude expensive providers

Architecture: Web layer only—CLI, Slack, other services couldn't access

I’d been using .env all along for my keys. It gets .gitignored and doesn’t go on the repository but I also never like to store sensitive data in the clear, and we need to get ready to support unique uses each with their own keys anyhow.

The goal: Production-ready LLM configuration before Alpha users.

Phase 0 investigation ran from 12:05 PM to 12:40 PM — 35 minutes mapping 17 files that used LLM clients, identifying security risks, analyzing cost patterns, and recommending a four-phase approach.

Then the implementation phases began.

Phase 1: Real API validation (90 minutes)

The first principle: Write tests first. True TDD.

Code agent created 28 tests covering:

Valid API keys for all four providers (OpenAI, Anthropic, Gemini, Perplexity)
Invalid keys properly rejected
Missing keys handled gracefully
Startup validation confirms all providers

Then watched them fail. All 28 tests: RED.

The critical decision: These tests make real API calls. No mocks for validation.

When you validate an API key against OpenAI’s servers, you need to actually call OpenAI. Mocking the response defeats the purpose. If the key is invalid or the API changed, you want to know immediately — not discover it later when a user hits that code path.

Implementation took 90 minutes. The tests revealed an immediate problem: Perplexity validation was failing. The agent had used model name “sonar” but Perplexity actually expected “llama-3.1-sonar-small-128k-online.”

Without real API calls, that bug would have shipped. The test suite would show green (mocked success) while production would fail (actual invalid model name).

By 1:52 PM: 26/26 tests passing. Four providers validated at startup. Real API calls confirming everything works.

Phase 2: Cost control (125 minutes)

The next problem: 87.5% of development tasks were using Anthropic. My personal API credits were burning during every development session.

I’ve been getting overage alerts for the past week or so.

The solution needed:

Environment-aware behavior (development, staging, production)
Configurable provider exclusion
Task-specific routing (general→OpenAI, research→Gemini)
Intelligent fallback chains

Implementation: 125 minutes for provider selection logic and 43 comprehensive tests.

The result:

# Development environment
PIPER_ENVIRONMENT=development
PIPER_EXCLUDED_PROVIDERS=anthropic
PIPER_DEFAULT_PROVIDER=openai
PIPER_FALLBACK_PROVIDERS=openai,gemini,perplexity

70% cost reduction in development — all general tasks now use OpenAI instead of Anthropic. Anthropic only gets used in production where cost is justified by quality requirements.

By 4:05 PM: Phase 2 complete, 43/43 tests passing.

Then my Chief Architect reviewed the work.

The architecture violation catch (4:59 PM)

At 4:59 PM, Chief Architect agreed with me that I had identified a critical issue. (Maybe “finally noticed” would be more accurate.)

The LLM configuration was attached to the web layer only. The initialization happened in web/app.py startup. This meant CLI commands, Slack integration, and other services couldn't access LLM configuration.

This violated our Domain-Driven Design patterns (documented in ADR-029 and Pattern-008). Domain services belong in the domain layer, not coupled to specific interfaces like the web layer.

The temptation (AIs love these kinds of shortcuts): Ship what works. The CLI and Slack integrations don’t use LLMs yet anyway. We could fix this later when it becomes a problem.

The discipline: Stop and fix the architecture now. Don’t ship 80% solutions.

The refactoring took 117 minutes across four phases of its own:

Phase 0 (6 minutes): Verify infrastructure — found 11 existing domain services with clear patterns to follow

Phase 1 (23 minutes with Serena): Create LLMDomainService and ServiceRegistry

Estimated: 2.5–3 hours
Actual: 23 minutes
92% faster than estimate

Phase 2 (12 minutes): Migrate 7 consumers to lazy property pattern

Phase 3 (36 minutes): Independent validation by Cursor — 7/7 architecture rules compliant

Was this 117-minute “delay” worth it? Thinking of it as “delay” misses the point. The point is not to ship broken code we will have to fix later at greater expense. It’s fine not to build something we don’t need yet, but it’s not OK to build it wrong now or allow an error to persist because it won’t cause problems yet.

The 117-minute refactoring delivered proper DDD architecture instead of web-layer coupling. If we’d waited until Alpha users needed CLI LLM access, fixing this would have taken days, not hours. We would have been refactoring under pressure with users depending on the broken architecture.

This is the inchworm principle in action: Don’t skip steps, even when the code works. Fix architecture issues immediately, not later.

By 7:45 PM: Architecture refactoring complete, validated by independent agent review.

Phase 1.5: Keychain security (71 minutes)

With proper architecture in place, the next layer: Remove plaintext API keys entirely.

The security upgrade:

Encrypted macOS Keychain storage
Migration tools with dry-run capability
Keychain-first priority with environment fallback
Helper methods for checking migration status

Three sub-phases:

Sub-Phase A — KeychainService (15 minutes):

241 lines of code
10 comprehensive tests
macOS Keychain backend verified
Estimated: 60 minutes
Actual: 15 minutes
75% faster

Sub-Phase B — Integration (63 minutes):

Keychain-first with environment fallback
Migration helpers for gradual transition
64/66 tests passing

Sub-Phase C — Migration CLI (5 minutes):

250 lines of migration tool with colored output
95 lines of API key validation script
Estimated: 50 minutes
Actual: 5 minutes
90% faster

By 9:21 PM: Migration tools complete. Time to test with real keys.

At 9:36 PM, I migrated my actual API keys to the macOS Keychain. The process worked flawlessly — keys moved from plaintext files to encrypted storage, backend started successfully, all four providers loaded from Keychain.

Then at 9:43 PM: Emergency. Backend wouldn’t start. “No LLM providers configured.”

The emergency fix (4 minutes)

Two methods were still checking config.api_key (from os.getenv) instead of get_api_key() (keychain-first pattern).

The inconsistency was obvious once identified. Most methods used the keychain-first pattern. These two didn’t. Fix took 4 minutes:

# Wrong (checking environment directly):
if self.config.api_key:z

# Right (keychain-first pattern):
if self.get_api_key():

By 9:48 PM: Backend starts successfully, all four providers load from Keychain, security upgrade complete.

The 4-minute emergency fix demonstrates why consistent patterns matter. Once the architecture is clear, deviations are obvious and quick to correct.

Phase 5: Documentation (2 minutes)

The final phase: Documentation for Alpha users.

Two comprehensive guides needed:

User setup guide (how to configure API keys)
Architecture documentation (how the system works)

Estimated time: 60 minutes for both guides. (Sure, Jan.)

Code agent completed both in 2 minutes.

97% faster than estimate.

The documentation ism’t shoddy, either. Both guides are comprehensive:

docs/setup/llm-api-keys-setup.md (186 lines)
docs/architecture/llm-configuration.md (243 lines)

Complete with:

Quick start instructions
Security best practices
Troubleshooting sections
Architecture diagrams
Migration guides

The Serena acceleration: Instead of reading through code files to understand what to document, instant semantic understanding of structure. Instead of manually finding all relevant files, find_symbol() returns complete references. Instead of validating completeness by scanning directories, get_symbols_overview() confirms all components covered.

By 9:45 PM: Documentation complete, 429 lines total, professional quality.

The post-push discovery (12 minutes)

At 9:56 PM, Cursor pushed all changes to GitHub and discovered: 15+ tests failing.

The keychain integration had broken tests that depended on environment variable mocking. Each test needed updates to properly mock keychain access instead.

This felt both like a bit of a failure (tests should have caught this earlier) but mostly just reality (integration changes sometimes reveal test gaps).

Cursor batch-fixed all affected tests in 12 minutes. Added proper keychain mocking, created a test specifically for keychain-first priority, verified all 42 LLM config tests passing.

By 10:08 PM: 42/42 tests passing, all changes committed, keychain integration complete.

The post-push test fixes weren’t a process failure — they were the final validation that the integration worked correctly. Better to discover test gaps immediately after push than have them lurk until someone touches that code again.

What the numbers mean

Thursday’s final accounting:

Code created: ~2,730 lines

1,550 lines of implementation
750 lines of tests
430 lines of documentation

Tests: 74/74 passing

Real API validation (no mocks)
Keychain integration tested
Provider selection validated

Security transformation:

Before: Plaintext .env file (HIGH risk)
After: Encrypted Keychain (production-grade)

Cost reduction: 70% savings in development (Anthropic excluded)

Architecture: DDD-compliant (proper domain layer)

Time invested: ~15 hours (5:35 AM — ~10:00 PM) in terms of duration but ultimately less than 90 minutes of my own focused attention.

But the real story is in the velocity comparisons:

With Serena:

Domain service: 23 minutes (vs 2.5–3 hours estimated) = 92% faster
Keychain service: 15 minutes (vs 60 minutes) = 75% faster
Migration CLI: 5 minutes (vs 50 minutes) = 90% faster
Documentation: 2 minutes (vs 60 minutes) = 97% faster

Four phases completed 75–97% faster than estimates.

This wasn’t agents rushing or cutting corners. The 117-minute architecture refactoring proved we weren’t sacrificing quality for speed. The 74 passing tests (including real API calls) proved functionality was solid. The A+ code quality rating (from next day’s audit) proved the work was production-ready.

The speed came from eliminating the exploration tax.

What comes next

Thursday ended with production-ready LLM configuration:

✅ Encrypted Keychain storage
✅ Real API validation at startup
✅ 70% cost reduction in development
✅ Proper DDD architecture
✅ 74 tests passing
✅ Comprehensive documentation

Sprint A1 progress: 2.5/4 issues complete. Two issues remained:

#216 (CORE-TEST-CACHE): Deferred to MVP milestone as part of #190 (MVP-TEST-QUALITY: Test Reliability for Production Confidence) — production cache works, test infrastructure polish not urgent
#212 (CORE-INTENT-ENHANCE): Improve intent classification accuracy — next Sprint A1 item

The plan: Complete #212 Friday, finish Sprint A1, move to Sprint A2.

But Thursday’s work set up something bigger. The Serena acceleration was infrastructure for everything that followed.

The 10⨉ multiplier is now operational. Every agent connected to both Claude Code and Cursor IDE. The semantic code understanding that eliminated exploration tax is available for all future work.

What we didn’t know Thursday evening: Friday would reveal gaps in the foundation we’d just celebrated completing. And Saturday, we’d use Thursday’s 10⨉ acceleration to fix those gaps faster than seemed possible.

But Thursday night, we’d just installed superpowers. And shipped production-grade security in a single day.

The methodology that enabled acceleration

The 92–97% speed improvements weren’t just Serena. They required the methodology that made proper use of the tool:

Phase −1 verification before starting: Confirmed infrastructure existed (11 domain services) before creating patterns from scratch

TDD with real API calls: Wrote tests first, confirmed failures, implemented features, confirmed success — catching Perplexity model name bug immediately

Architecture review at critical points: Chief Architect intervention at 4:59 PM prevented shipping web-layer-coupled LLM config

Independent validation: Cursor verified DDD compliance (7/7 rules) without knowing Code agent’s implementation details

Consistent patterns throughout: Lazy property pattern for module singletons, keychain-first priority everywhere, comprehensive error handling

The tool provided the capability — semantic code understanding, instant pattern discovery, zero exploration tax. The methodology provided the discipline — verify before building, test before implementing, review architecture, validate independently.

Neither works without the other. Serena without methodology: Fast but brittle implementations. Methodology without Serena: Slow but solid implementations.

Together: Fast AND solid.

What Thursday teaches

The exploration tax is real. Before Serena, agents spent 15–20 minutes reading files to understand patterns before writing any code. That overhead compounded across every feature, every refactoring, every architectural decision.

Eliminating that tax didn’t just make work 15–20 minutes faster. It made work an order of magnitude faster by enabling agents to focus on implementation without simultaneously processing hundreds of lines of context.

But Thursday also teaches that acceleration without discipline is dangerous. The 92–97% speed improvements were only valuable because:

Tests were comprehensive (74 passing, real API calls)
Architecture was reviewed (caught web-layer coupling)
Quality was verified (independent validation)
Patterns were consistent (lazy properties, keychain-first)

Speed is often presented as a tradeoff with discipline. This is a false choice. You need both. Fast implementations without quality create technical debt that slows future work. Quality implementations without speed miss opportunities when timing matters.

Thursday delivered both: Production-grade security in six hours. 70% cost reduction. Proper DDD architecture. 74 passing tests. Comprehensive documentation.

And the infrastructure to make everything that followed possible.