The Wizard that Learned to Restart Itself

The Wizard that Learned to Restart Itself

October 29

Wednesday morning, 5:58 AM. One day before we plan to onboard our first non-me alpha user. Time for the real test: Can someone actually run python main.py setup and get Piper Morgan working?

Not “can they follow documentation?” Tuesday proved the docs had seven blockers. Fixed five. Deferred two. Documentation now mostly correct.

But documentation isn’t the system. The wizard is.

The setup wizard that’s supposed to:

• Check prerequisites automatically
• Install dependencies in a fresh venv
• Start Docker services
• Create database tables
• Set up user account
• Configure API keys

Tuesday tested documentation. Wednesday tests the actual automation. By 8:55 AM, three critical bugs discovered. Each one a complete blocker. Each one requiring systematic fixes.

By day’s end: All three bugs fixed. Wizard systematically hardened. Seven system checks implemented (Docker, Python, Port, PostgreSQL, Redis, ChromaDB, Temporal). Multi-service startup automated. Smart detection of what’s running and what needs starting.

Bug 1: The Docker service name nobody checked

6:53 AM. First wizard run begins.

User follows updated Tuesday documentation. Reaches Docker setup step.

Guide says: docker-compose up -d db

Error message: ERROR: No such service: db

Docker Compose looks at docker-compose.yml. Service is defined as postgres, not db.

This is what happens when LLMs guess at syntax instead of checking first.

Simple error. But complete blocker. User cannot start database. Cannot proceed.

Fix 1: Update documentation to docker-compose up -d postgres

Fix 2: Update wizard checks to reference correct service name

Fix 3: Add explicit "Launch Docker Desktop" step with visual indicators

This reveals deeper issue: Docker Daemon not running.

User runs docker-compose without launching Docker Desktop first. Error:

Cannot connect to Docker daemon at unix:///var/run/docker.sock

The guide assumed: “Of course Docker Desktop is running.”

Reality: Fresh laptop, Docker Desktop installed but not launched, user follows commands exactly.

Added explicit step: “Launch Docker Desktop (look for whale icon in menu bar).”

The DRY principle applied to documentation

7:08 AM. While fixing Docker issues, notice another problem: Prerequisites duplicated everywhere.

Problem: step-by-step-installation.md has Checks 1-4 (177 lines) duplicating content in PREREQUISITES-COMPREHENSIVE.md.

Two sources of truth. Double maintenance burden. Guaranteed drift over time.

Solution: Remove 177 lines from step-by-step. Add link to comprehensive guide instead.

Correct flow now:

1. README.md → tells you which doc to read
2. PREREQUISITES-COMPREHENSIVE.md → verify you have everything
3. step-by-step-installation.md → follow installation (no redundant checks)

Single source of truth. Easier maintenance. No contradiction possible.

The systematic thinking emerging: Not just fixing individual bugs. Fixing patterns that create bugs.

Bug 2: The chicken-and-egg problem

7:44 AM. The critical bug. The one that required genuine architectural thinking.

User runs python main.py setup to start wizard.

Wizard’s intended flow:

1. Check system prerequisites
2. Create fresh venv at .venv/
3. Install requirements.txt into venv
4. Check database connectivity
5. Create user account
6. Configure API keys

What happens at step 4:

ModuleNotFoundError: No module named 'sqlalchemy'

But we just installed sqlalchemy in step 3! It’s in requirements.txt. Installation succeeded. So why can’t wizard import it?

Root cause — The chicken-and-egg problem:

1. User runs python main.py setup (uses their current Python environment)
2. Wizard creates fresh venv + installs all requirements into that venv
3. Wizard keeps running in original Python environment
4. When wizard tries database check → imports sqlalchemy → FAILS
5. sqlalchemy exists only in the new venv, not in original Python

The wizard created an environment it can’t use because it’s not running in that environment.

First attempt: Remove database check

7:47 AM. Initial fix attempt.

Solution: Remove check_database() from system checks. Skip validation. Proceed without checking.

Result: Different error.

ModuleNotFoundError: No module named 'structlog'

Still failing. Still trying to import from venv while running in original Python.

Problem isn’t database check. Problem is wizard can’t access the dependencies it just installed.

Real fix: Wizard restarts itself

7:56 AM. The actual solution.

The insight: Wizard needs to restart itself in the venv it creates.

Implementation:

1. Wizard detects if already running in venv
2. If not in venv: Create venv, install requirements, restart itself using venv Python
3. Use os.execv(venv/bin/python, [python, main.py, setup])
4. Process is replaced — wizard continues execution but now in venv with all dependencies

Result: Wizard runs in the venv it creates. Can import everything. All checks work. Elegant!

7:59 AM. Bonus achievement: Restore database check.

Since wizard now runs in venv, it can import sqlalchemy. Database check restored to system checks.

All 4 checks now active:

• Docker running
• Python 3.9+
• Port 8001 available
• Database accessible

Complete system validation before user creation!

Bug 3: The port mismatch cascade

8:30 AM. Database check running. New error:

Database check details: [Errno 61] Connect call failed ('::1', 5432, 0, 0)
✗ Database accessible

Wizard trying to connect to port 5432 (PostgreSQL default).
 Piper Morgan uses port 5433 (from docker-compose.yml).

We are regressing to bugs I’ve been fighting since day one. In the codebase and methodology we have learned to specify port numbers and other non-default config settings but somehow all that went out the window when we wrote these docs.

Root cause: services/database/connection.py line 70:

port = os.getenv("POSTGRES_PORT", "5432")

No POSTGRES_PORT environment variable set → defaults to wrong port!

First fix: Wizard sets POSTGRES_PORT=5433 before system checks.

But this is bandaid. Real problem: Code defaults don’t match Docker config.

The systematic audit

8:38 AM. My feedback: “Please do not populate the wizard with generic guesses. All of this information is documented and available. Please do a cross-comparison between the wizard’s logic and the docs.”

Cursor performs systematic audit:

docker-compose.yml

Has correct settings:

• Password:dev_changeme_in_production ✅
• Port: 5433 ✅

services/database/connection.py

Has incorrect, generic settings, never fixed:

• Password: dev_changeme ❌
• Port: 5432 ❌

scripts/setup_wizard.py

A band-aid:

Password: Inherits from code ✅

Port: Override to 5433⚠️

Three different configurations. None fully matching the others.

Systematic fix applied:

1. Fixed services/database/connection.py defaults to match docker-compose.yml
2. Removed wizard bandaid (now uses correct code default)
3. Fixed .env.example to document correct values

Result: Code, Docker, wizard, AND .env.example all aligned.

Not “fix the immediate error.” But “fix the pattern that creates errors.”

Bug 4: The missing database schema

8:46 AM. Database connectivity verified. Wizard proceeds to user creation.

❌ Setup failed: relation "users" does not exist
[SQL: INSERT INTO users ...]

The wizard checked database was accessible. But never created the tables.

scripts/init_db.py exists for table creation. Wizard jumped straight to user creation without running it.

Fix: Add “Phase 1.5: Database Schema” step.

Implementation:

1. Check if tables exist (SELECT 1 FROM users)
2. If not, call db.create_tables() (creates all models)
3. Idempotent — won’t recreate if tables exist

Flow now:

1. System checks (Docker, Python, Port, Database connection)
2. Database schema creation ← NEW!
3. User account creation
4. API keys setup

The “stop being reactive” moment

8:55 AM. Three bugs found. Three bugs fixed. But my feedback:

“I still feel we are using a naive process here vs. a planned one. We should have known we’d need database tables. Can you possibly anticipate other steps the wizard may not yet be including?”

Because by now it dawned on me that in attempting to write a little setup script I have completely abandoned all flywheel discipline. Where is the architectural planning leading to a crisp gameplan? Where are the strict prompts? The moment I decide something is “straightforward” and strat winging it again is it really a surprise to find methodology regressions from literally months ago cropping up again immediately?

This is the turning point. Stop finding bugs reactively. Start thinking systematically.

Cursor creates comprehensive audit: wizard-completeness-audit.md

Critical finding: Wizard checks 1 of 5 Docker services:

• ✅ PostgreSQL (5433) — Only one checked!
• ❌ Redis (6379) — NOT CHECKED
• ❌ ChromaDB (8000) — NOT CHECKED
• ❌ Temporal (7233) — NOT CHECKED
• ❌ Traefik (80) — NOT CHECKED

Missing phases:

• ❌ Phase 4: Configuration (PIPER.user.md, .env)
• ❌ Phase 5: Service Verification (E2E test)
• ❌ Phase 6: Post-Setup (summary, next steps)

My decision: “This is all work we were going to have to do at some point and this is exactly the right time to do it.”

Not “fix these three bugs and call it done.” But “make the wizard complete and systematic.”

The systematic implementation

9:00 AM. Comprehensive wizard hardening begins.

Part 1: Multi-service checks

Implemented:

1. check_redis() - Redis connectivity (port 6379)
2. check_chromadb() - ChromaDB connectivity (port 8000)
3. check_temporal() - Temporal connectivity (port 7233)
4. Updated check_system() to check ALL 7 requirements:

• Docker installed
• Python 3.9+
• Port 8001 available
• PostgreSQL (5433)
• Redis (6379)
• ChromaDB (8000)
• Temporal (7233)

Smart service startup

Added start_docker_services():

• Detects which services are down
• Automatically runs docker-compose up -d
• Waits for services to be ready
• Verifies all 4 services accessible
• Timeout protection (2min initially)

Smart flow:

1. Check which services are running
2. If some are down, offer to start them automatically
3. Run docker-compose only if needed
4. Re-check services after starting
5. Provide troubleshooting if still failing

The timeout problem

10:22 AM. Testing the smart startup on fresh laptop.

🐳 Starting Docker services...
   (This may take a minute on first run)
   ✗ Timeout waiting for services to start

Root cause: First-time Docker image pulls can take 5–10 minutes.

Images being pulled:

• postgres:15
• redis:7
• chromadb
• temporal

Combined size: ~2GB

Timeout was 120 seconds (2 minutes). Docker images hadn’t finished downloading before wizard gave up.

Fix applied:

1. Increased timeout: 120s → 600s (10 minutes)
2. Changed to Popen with live feedback
3. Progressive health checks (30 attempts × 2s = 1 minute per service)
4. Progress messages every 10 seconds: “2/4 services ready…”
5. Better UX messaging:

• “First run may take 5–10 minutes to download images”
• “Pulling and starting containers…”
• Shows which services are ready/not ready

Result: User sees progress. Won’t timeout during image downloads. Understands what’s happening.

Making Temporal optional

3:50 PM. Testing complete wizard flow.

✓ PostgreSQL
✓ Redis
✓ ChromaDB
✗ Temporal (7233) - Timeout

Three of four core services working perfectly! But Temporal failing.

Analysis:

• Temporal is known to be flaky
• Temporal is NOT required for user setup or account creation
• Should not block wizard from continuing
• Most features work without Temporal

I asked the chief if making Temporal optional rather than required was a pragmatic alpha decision or papering over infrastructure issues, and was reassured this will be OK. I recommend we attempt to continue lazy-loading Temporal in the background without allowing it to stop the process.

Fix: Made Temporal optional. Won’t block setup. Wizard warns if unavailable but continues.

User can complete setup. Create account. Start using Piper Morgan. Temporal can be fixed later without blocking alpha.

What Wednesday achieved

Let’s measure Wednesday properly.

Bugs found: 3 critical (Docker service name, chicken-and-egg, missing schema)

Bugs fixed: 3 (all same day)

System checks implemented: 7 (Docker, Python, Port, 4 services)

Services automated: 4 (PostgreSQL, Redis, ChromaDB, Temporal)

Documentation updates: Multiple (Docker steps, prerequisites, service names)

Architectural improvements: Wizard restart pattern, systematic validation

Time: 11h 36m (5:58 AM — 5:34 PM)

More importantly: Philosophy shift

Before Wednesday: Reactive bug fixing. Find problem → Fix problem → Move on.

After Wednesday: Systematic completion. Anticipate gaps → Fill proactively → Verify comprehensively.

The “stop being reactive” moment at 8:55 AM changed everything. Not just fixing three bugs. But making wizard complete.

The systematic wizard now works

By Wednesday end of day, wizard flow is:

Phase 0: Environment Setup

1. Detect if running in venv
2. If not: Create venv, install requirements, restart in venv
3. Now running in proper environment with all dependencies

Phase 1: System Checks

1. Docker installed and running
2. Python 3.9+ available
3. Port 8001 not in use
4. PostgreSQL accessible (5433)
5. Redis accessible (6379)
6. ChromaDB accessible (8000)
7. Temporal accessible (7233, optional)

Phase 1.5: Service Startup

1. Detect which services are down
2. Offer to start Docker Compose automatically
3. Wait for services with progress indicators
4. Handle first-time image downloads (10min timeout)
5. Verify all services ready before proceeding

Phase 2: Database Schema

1. Check if tables exist
2. If not, create all tables
3. Idempotent, safe to re-run

Phase 3: User Account

1. Create initial user
2. Set credentials
3. Configure permissions

Phase 4: API Keys

1. Guide through keychain setup
2. Verify keys configured
3. Test connectivity

What the three bugs taught us

Bug 1 (Docker service name): Test your documentation. Don’t assume service names. Verify exact commands.

Bug 2 (Chicken-and-egg): Environment creation is complex. Wizard needs to work in the environment it creates. Sometimes that requires restarting itself.

Bug 3 (Missing schema): Don’t assume prerequisites exist. Verify explicitly. Create what’s missing.

But the real lesson isn’t about these three specific bugs.

The real lesson: Stop being reactive. Be systematic.

When you find one bug, don’t just fix that bug. Ask: “What category of problem is this? What other instances of this category exist? How do we prevent this category entirely?”

Wednesday’s bugs:

• Revealed patterns (configuration mismatches, missing validations, assumption failures)
• Drove systematic fixes (audit all configs, check all services, verify all prerequisites)
• Transformed approach (reactive → systematic)

Thursday morning confidence

Wednesday ends with wizard systematically hardened. Seven checks. Four services automated. Smart startup detection. Progress indicators. Comprehensive validation.

Thursday morning… I resume testing as the first alpha user. I need to inform Beatrice we need another week!

Can I run python main.py setup now from scratch and get Piper Morgan working on my clean laptop?

After Wednesday’s work: Yes. Probably. Hopefully.

The wizard that learned to restart itself. The systematic checks. The smart service startup. The comprehensive validation.

Three bugs found Wednesday. Three bugs fixed Wednesday. But more important: Philosophy transformed Wednesday.

Stop being reactive. Start being systematic. Anticipate gaps. Fill proactively. Verify completely.

Thursday will test whether Wednesday’s systematic transformation was enough.