What AI Progress Feels Like in Practice
Exponential change does feel exponential.
Last Thursday I stood in front of 20 top executives from a major clothing retailer in the UK, delivering a training session on AI. What I told them might have been right last week, but not anymore.
I walked them through the architecture of one piece of AI infrastructure at Mumsnet that is responsible for complex data analysis. I explained how AI isn’t capable of producing an accurate answer in this particular case, and how we’re solving it with 13 revision agents that double-check individual claims.
So yesterday Anthropic released Fable, a new AI model. Today, I asked it to do the same complex work in one shot without extra guidance or expensive verification that used to catch lots of errors.
It did a better job in about 10% of the time, burning fewer tokens, and, crucially, without me having to create that very complex AI skill that used to fix complex mistakes.
Intellectually, I get it. That’s what exponential progress looks like, that’s what was supposed to happen. Maybe in April I should have just waited instead of working around Claude’s then-limitations.
But it still feels like magic. When I did the run on Fable to compare and found out that a far simpler approach produced a far better result, I swore out loud. I was alone working from home :)
The conclusion I’m taking from this is that if I find myself doing work that AI could or should reasonably be doing, I’m probably doing it wrong.
If AI is smart enough, has access to the data it needs, the task is clearly formulated and the context is set correctly, it should succeed.
And if it doesn’t, it probably will soon enough.
Interested in doing this kind of work together?
I’m hiring an Agentic Orchestration Engineer at Mumsnet. We’ll be working together on building AI infrastructure here. If you’re interested in this job, tell me how would you approach building AI skills in a constantly evolving environment at evgeny.shadchnev@mumsnet.com.
At the high level the job is to take Mumsnet into the AI-first future, whether it means building AI agents, training the team, changing their data architecture or choosing the best AI tech providers.
More specifically, in the near-term, the business needs to build a set of AI skills, agents and MCP servers that will help us better leverage the human talent we have, enabling them to focus on what only people are good at. You and I will be driving AI transformation there.
What I’m looking for:
Experience building AI skills and agents, managing the context skilfully. We’re working with both Claude and ChatGPT/Codex.
Keen interest to learn fast and move fast. We’ll be iterating and experimenting.
An aspiration to build systems operating at levels 4 and 5 of this framework. If we’re reviewing code for QA, we’re not advanced enough.
Probably 1-3 years of dev experience, but if you never had a developer job and taught yourself how to use Claude Code or Codex at a sophisticated level, please get in touch and share the details.