The Three Questions Every AI Builder Should Ask

July 22

Eight weeks into this project I was staring at a roadmap that claimed my AI assistant was nowhere near ready for real work, while simultaneously reading reports of it coordinating a complex technical sprint with surgical precision. Something didn’t add up.

That’s when I realized I needed to have “The Conversation” with my Chief Architect about readiness milestones. Not the usual “is the code working?” check-in, but the strategic inflection analysis that every AI builder eventually faces.

Fast forward to now: This conversation preceded what became the Great Refactor by six weeks. I was already sensing the gap between documentation and reality that would become a central theme of the march toward alpha, but NO SPOILERS.

The documentation reality check

First, the awkward discovery. Our roadmap was showing completed features as “not started.” PM-010 (error handling) and PM-011 (web UI) were marked as future work, but they’d been done for weeks. The session logs told the real story, but somehow the official docs had drifted four to six weeks behind reality.

This is embarrassingly common in AI development (really, kinda all development? especially if nobody is really reading the docs), where rapid iteration can outpace documentation faster than traditional software. But it also revealed something important: we were in some ways much closer to useful than we thought.

OK, today speaking again: The “documentation lagging reality” pattern would become the core insight of September’s work. What I called a minor discovery in July was actually the first glimpse of something we’re now calling “the 75% pattern” that nearly blew up the QueryRouter in September. OK, I’m hushing.

The three universal questions

Every AI builder — whether you’re creating a coding assistant, a customer service bot, or a product management companion — eventually faces three critical timing questions:

1. When does it become useful?

The real question: When can you start dog-fooding your own creation for actual work?

For Piper Morgan, this meant: when can I use it for genuine PM tasks, even if it’s not perfect? Not “when is it feature-complete,” but “when does it cross the utility threshold?”

Our answer at the time: 1–2 weeks away (early August 2025)

The gap: Just PM-012 (real GitHub issue creation). Everything else — error handling, web UI, intent classification, content search with 642x performance improvement — was already working.

The insight: We’d been conflating “feature-complete” with “minimally useful.” Those are very different milestones.

Two months later here: This optimism was… premature. QueryRouter would be disabled by August and require systematic resurrection in September. But the insight about utility vs completion proved crucial for the Inchworm Protocol. What was true was that we could have a single end-to-end feature working in that time, but apparently not without bypassing some of our strict models. OK, anyway…..

2. When can it start learning?

The real question: When do you begin the parallel track of education alongside development?

This isn’t about machine learning models getting retrained. It’s about when you start systematically feeding your AI context about your domain, your preferences, your decision patterns. When does it make sense to begin the “teaching” phase while you’re still building core features?

Our answer: 2–3 weeks (mid-August 2025)

The preparation: Already happening through session logs, architectural decisions, and process documentation

The active phase: When PM-043 (feedback processing) and PM-044 (clarifying questions) get implemented

The insight: Every session log I’m creating IS educational material. The very act of building Piper thoughtfully is teaching her excellent PM practices.

Two months later: The session logs indeed became the foundation for briefing documents and methodology. Teaching through building proved more valuable than formal education modules. The phase of actively “Educating Piper Morgan” is still a few refactors and some other core features way from even really starting.

3. When does it become self-improving?

The real question: When can your AI start contributing to its own development process?

This is the bootstrap moment — when your creation becomes a team member rather than just a tool. For an AI PM assistant, this means creating GitHub issues about her own bugs, prioritizing her feature requests, tracking her sprint progress.

Our answer: Stage 1 in 3–4 weeks (late August 2025)

The stages:

Stage 1: Self-reporting (tracking success rates, identifying knowledge gaps)
Stage 2: Active participation (creating tickets, generating documentation)
The bootstrap moment: When Piper creates her first GitHub issue saying “Intent classification failing for financial terms — need pattern update”

The insight: This isn’t just a technical milestone — it’s an architectural philosophy. Design for eventual self-management from day one.

Two months later: Self-improving AI remains aspirational, but the multi-agent coordination we developed exceeded these early visions. Agents now create gameplans, cross-validate each other’s work, and systematically debug complex issues. Piper isn’t really directly involved and won’t be actively even administering its own project for at least a month if not more.

The timeline compression discovery

Here’s what surprised me: the analysis revealed we weren’t months from the first threshold of meaningful utility — we were weeks away. The gap between “building AI” and “using AI” can be smaller than expected, but only because we’d been systematically addressing the right foundational issues.

The compound benefits were accelerating everything, not so much making us work faster but with less wasted motion and fewer wild bug chases. Python 3.11 enables modern async patterns. Test reliability enables confident changes. Configuration patterns enable clean architecture. Each systematic improvement makes the next one faster.

Two months later butting in again now: Timeline compression proved illusory in July but real by September. The systematic improvements were indeed compounding — they just needed the Inchworm Protocol to unlock their potential.

A meta-learning opportunity

But here’s the really interesting part: those three questions aren’t just about planning — they’re about recognizing that the development process itself is training data.

Every architectural decision I make, every sprint I run, every bug we systematically fix is part of teaching Piper how to be an excellent PM. It’ll learn not just from ingested books and frameworks, but from watching systematic product management in action.

When Piper eventually starts managing its own development, it’ll have insights derived from hundreds of hours of observing thoughtful technical leadership (ahem). That’s a very different kind of AI education than just feeding it PM bestsellers.

Two months later: This meta-learning insight proved prescient. While not baked yet into Piper’s own multi-agent orchestration modules yet, the briefing documents and methodology templates that we’ve refined heading into the Great Refactor embody exactly this — systematic PM practices distilled into reusable patterns.

The strategic inflection framework

So here’s what I learned: these three questions create a framework for strategic inflection analysis in AI development:

Question 1 forces honest capability assessment. Strip away the roadmap aspirations and ask: what percentage of target tasks can it handle right now?

Question 2 surfaces the parallel track opportunity. You don’t have to wait for feature-complete to begin domain education. Start now with the materials you’re already creating.

Question 3 designs for eventual autonomy. Build metrics collection, self-diagnostics, and pattern recognition from day one, so the bootstrap moment becomes natural evolution rather than forced retrofit.

Two months later sliding into your DMs again here: This framework guided the evidence-based approach that characterized September’s systematic completion work. Question 1’s “honest capability assessment” became the foundation for rejecting mocked success in favor of real performance.

The activation month

Based on this analysis, we’re not too far from “Piper Activation Month” — the transition from development project to active PM assistant. The Foundation Sprint was the beginning of a long process of smoothing the roadway we are systematically walking down now.

At the time I wrote this we thought it might be August. Here at the end of September it’s clear we still have a lot of finishing to do, but it’s definitely in sight now. Two months later out.

The universal pattern

Whether you’re building a coding assistant, a customer service bot, or a product management companion, these three inflection points are universal:

When does it cross the utility threshold? (Usually sooner than you think if you’re solving real problems)
When do you start the education parallel track? (Usually sooner than you plan, using materials you’re already creating)
When does it start contributing to itself? (Usually in stages, with self-awareness coming before self-management)

The key insight: these aren’t just development milestones — they’re strategic planning tools. They force you to distinguish between “feature-complete” and “minimally useful,” between “AI tool” and “AI team member.”

And sometimes, if you’re lucky, they reveal you’re closer than you thought.