Where AI Should Sit in Your Org Chart
Three structures, each with a predictable failure mode. The right one depends less on strategy than on how many people you have.
Erin Moore
Fractional Chief AI Officer
There are three ways to structure AI in an organisation — a central team, specialists embedded in each function, or a hybrid with a small core and embedded practitioners. Each has a predictable failure mode, and the right choice depends far more on your headcount than on your strategy.
Central team
One group owns AI work for the whole company.
Good at: consistency, governance, avoiding duplicate tooling, building genuine depth.
Fails by: becoming a bottleneck and drifting away from the actual business problems. The classic symptom is a central team with a six-month backlog and departments quietly buying their own tools to route around it — manufacturing the shadow AI the structure was meant to prevent.
Embedded specialists
AI capability sits inside each function, reporting to that function's leader.
Good at: relevance and speed. Solutions fit the real work because the person builds alongside the people doing it.
Fails by: duplication and divergence. Three departments solve the same problem three ways, nobody owns governance, and the total spend is invisible until someone adds it up.
Hybrid
A small central core owning standards, governance, vendor selection and measurement, with embedded practitioners doing the work in each function.
Good at: most things, which is why most organisations converge here eventually.
Fails by: ambiguity about who decides. The hybrid only works if the split is written down — typically the core owns what and whether, the function owns how.
What actually determines the answer
Under ~50 people: none of the above. One accountable person and a policy. Structure is premature.
50–250: hybrid, with the "core" often being a single fractional Chief AI Officer plus whoever is doing the work in each function.
250+: hybrid with a real core team, or central if governance risk dominates.
The mistake is adopting a structure that suits an organisation four times your size because that is what the case studies describe.
The part that survives any structure
Whatever the shape, three things need a named owner: the approved-tools list, the register of what is running, and the measurement baseline. NIST's AI Risk Management Framework puts "govern" first for exactly this reason — accountability precedes architecture. If you cannot name who holds those three, the org chart is decorative.
Frequently asked questions
Should AI be centralised or embedded? Most organisations end up hybrid: a small core owning standards, governance, vendor choice and measurement, with practitioners embedded in each function doing the work.
What size company needs an AI team at all? Below roughly 50 people, a named owner and a one-page policy beat any structure. Team design matters from a few hundred people upward.
What is the failure mode of a central AI team? Becoming a bottleneck. A long backlog pushes departments to buy their own tools, which reproduces the fragmentation the central team existed to prevent.
Who should own AI governance in a hybrid? The core, not the functions. Governance split across functions diverges quickly, and nobody is answerable when it does.
Further reading
Tagged with:
Ready to Automate Your Business?
Let's discuss how AI automation can deliver measurable ROI for your organization in 90 days or sooner.