The Great Filter: why AI will break your business (and your product career)
We barely survived the rigid logic of enterprise software. Now comes software that works on probabilities. For the unprepared, this ends badly.
There's an old joke in tech: if you want to make a quick buck, short any company that announces a massive ERP rollout.
It's funny because it's true. For twenty years we've watched organisations bleed time and money getting these giant systems live. An ERP demands precision. It forces you to map the exact physics of how your business actually works — and most companies, honestly, can't describe that.
If we couldn't handle the rigid logic of ERPs, we are in no way ready for software that works on probabilities.
And that's what's arriving. Deploying AI isn't an office-software upgrade; it's structural surgery on roles, data pipelines and team dynamics. The market is already splitting into three camps. For product managers watching it happen, the message is uncomfortable: build real technical depth, or get left behind.
Three ways this plays out
1. The architects (AI-first)
An exclusive club. These companies aren't bolting a chatbot onto the homepage; they're building or deeply customising their own models and reshaping the business around them. They're also the few that actually solved the operational-clarity problem that killed their old software projects.
2. The pragmatists (augmentation)
This group survives, and probably thrives, through smart partnerships. Rather than building from scratch, they lean on OpenAI or Google and pour their energy into retrieval-augmented generation — safely feeding their own company data into the models. It won't reinvent the business overnight, but it makes their teams measurably faster.
3. The tapestry weavers (everyone else)
Most companies will land here, and most will regret it. They'll tip their raw data into a generic LLM and pray. What comes back won't be a competitive edge. It'll be a product that sounds exactly like everyone else's.
The tapestry trap
Lean on a generic model without guardrails or unique context and the output isn't clever — it's lukewarm. Two warning signs give it away:
- Surface-level fluff. Features that spit out bulleted summaries full of empty buzzwords, vaguely crediting unnamed “industry observers.”
- The “delve” epidemic. Linguistic copy-paste. If your app's automated insights are forever “delving” into “intricate tapestries,” you haven't built a feature. You've built a wrapper.
Why this threatens PMs specifically
If the risk for businesses is blending into the background, the risk for product managers is obsolescence. Vibe coding is here: engineers are describing systems in plain language and watching them appear in minutes. The old barriers to building software are gone.
// The technical reality check
You have to understand the shift from deterministic systems (predictable databases) to probabilistic ones (moody LLMs). If you don't know why a model hallucinates a number range or breaks its own formatting, you cannot safely manage the product.
The way through
Here's the short version. If your company can't describe its own operations clearly enough to pass a software audit, it has no chance of surviving an AI transformation.
The winners of the next decade won't be the executives pointing at vision decks. They'll be the builders — the ones who understand the machinery well enough to take it apart.