August 20
After months of building with multiple AI agents, a pattern keeps emerging: We create sophisticated systems, lose track of what we built, then rediscover our own achievements through “archaeological” investigation.
This recurring cycle of institutional amnesia may be a bug in our process but for today’s LLM services, it’s a feature that reveals the real UX challenge ahead.
The intelligence plateau and the orchestration valley
The AI industry is obsessed with reasoning capabilities. Larger context windows, better chain-of-thought, more sophisticated inference. Meanwhile, anyone actually building with AI faces a different problem entirely: How do you coordinate multiple specialized capabilities without losing your mind?
Anyone reading this series has the right to question what this process may be doing to my mind at this very moment!
Yesterday we discovered 599 comprehensive smoke tests we’d apparently built and then completely forgotten. Saturday we rediscovered attribution systems we’d implemented but lost track of (in fact, I only just now remembered it again and added it to my notes to include ATTRIBUTION.md to our weekly doc sweep). Two weeks ago we found enterprise-grade feedback APIs sitting in our codebase, unmarked and uncredited.
The pattern isn’t forgetfulness — it’s that our tools for building are ahead of our tools for remembering.
From brilliant generalists to orchestrated specialists
The current paradigm assumes one brilliant AI that can handle anything you throw at it. The emerging paradigm recognizes that specialized tools, properly coordinated, deliver better results than generalist intelligence.
Our accidental prototype:
- Claude Code: Architecture and systematic implementation
- Cursor Agent: Targeted debugging and focused fixes
- Chief of Staff: Coordination and strategic oversight
- Chief Architect: Decision-making and system design
Each agent has different context levels, different strengths, different appropriate use cases. The magic isn’t in making any individual agent smarter — it’s in the orchestration patterns that let them work together effectively.
One thing this enables me to do is to have focused coherent conversations and decision-making processes always at the right level of abstraction. Early on I found that as soon as multiple contexts get mixed you get a mishmash of more generic and sloppy advice and results. It’s kind of like how if you mix too many paints you end up with the same muddy brown.
The UX we actually need
After coordinating multi-agent workflows for months, I’m realizing that the UX challenges aren’t about reasoning — they’re about:
- Context handoffs: How do you maintain working memory across agent transitions?
- Coordination protocols: How do you deploy the right agent for the right task without overwhelming the human orchestrator?
- Institutional memory: How do you prevent the “forgotten monuments” cycle where sophisticated systems get lost in your own complexity?
- Verification workflows: How do you maintain quality when multiple agents contribute to the same outcome?
Each of these is critical and urgent in its own way. Getting any of these wrong means you are just injecting chaos into your processes.
Throwing intelligence at everything
We keep applying intelligence solutions to orchestration problems. Need better coordination? Train a smarter model. Need better memory? Increase context windows. Need better task routing? Build more sophisticated reasoning.
Except, orchestration isn’t really an intelligence problem. It’s a UX design problem.
My failed adoption of the TLDR system is a perfect illustration. I absorbed something that sounded cool to me without really understanding it was intended to work with 50ms test timeouts from compiled languages, which ignores Python’s ecosystem realities. More intelligence wouldn’t have fixed the fundamental mismatch where understanding my constraints better would have.
Affordances over algorithms
UX for AI will be defined by:
Specialized models over generalist LLMs. A focused SLM that understands database schemas will outperform a brilliant generalist that has to reason about every query from first principles.
Orchestration patterns over individual agent capabilities. The system that deploys the right specialist at the right time beats the system with the smartest individual components.
Context management over context windows. Better handoff protocols matter more than larger memory capacity.
Coordination affordances over reasoning power. Tools that help humans orchestrate AI workflows effectively will matter more than tools that make individual AI agents more capable.
I can’t even say how these affordances will look or behave. I’m treading the cowpaths now, and hoping talented UX designers (hey, I’m just a PM these days!) can figure this out and save me all the manual work and cognitive labor I do to provide resilience and coherence via scaffolding, harness, redundancy, and other the other hacks I’ve been picking up through trial and error (and stealing ideas from other people!).
The working memory revolution
Our recurring “archaeological discovery” pattern reveals the real frontier: building systems that maintain institutional memory across time, people, and context switches.
Every time we rediscover forgotten excellence, we’re experiencing the same challenge every team building with AI will face: How do you scale human-AI collaboration without losing track of what you’ve accomplished?
Orchestration as a new kind of literacy
Pretty soon, prompting individual AI agents effectively will stop being the valuable skill (or parlor trick) it is today. What we’re going to look for is the ability to orchestrate multiple specialized AI capabilities without losing coherence.
Product managers will need orchestration patterns for coordinating AI-augmented workflows across teams.
Designers will need to make (and use!) affordances for human-AI collaboration that maintain user agency while leveraging AI capabilities.
Engineers will need architecture patterns for composing AI services without creating coordination overhead.
The Piper Morgan thesis
While I am definitely building a product management tool, I find I am also prototyping the UX patterns that are like to define human-AI collaboration, or at least point us in the right direction, over the next decade.
I always knew this was a learning project. I sincerely want ship v1 of Piper Morgan and deliver value to myself and ideally others as well. At the same time it’s been incredibly rewarding just plunging in learning things constantly, and then turning around quickly to share my enthusiasm with all of you.
What I didn’t realize is that beyond building Piper Morgan, I may be studying just exactly the sort of interesting puzzles and problems and opportunities that the brightest minds in UX and digital software product development need to be figuring out, and fast! (Before the bad guys own it all.)
My recurring cycle of building sophisticated systems, losing track of them, and rediscovering them through archaeological investigation provides some ongoing comic relief for anyone following along, as well as an endless rollercoaster ride of elation and chagrin for me, and it also happens to be one of the fundamental challenges that every organization building with AI will face.
Smarter AI isn’t going to get us there, but better orchestration just might.