Chief AI Officer KPIs

Measure a Chief AI Officer on portfolio return, decision quality, governance completeness, cancellation rate and organisational capability — not on how many AI projects launched. Counting launches rewards starting things, which is the opposite of what the role exists to do.

The five

1. Portfolio return against stated baselines. The aggregate of what was funded, measured against baselines committed before starting. Aggregate matters: individual wins are easy to cherry-pick.

2. Decision quality, reviewed retrospectively. Once a quarter, revisit decisions made two quarters ago. Were they right given what was knowable then? This is the closest honest measure of judgment, and it tolerates bad outcomes from good decisions.

3. Governance completeness. Is the register current, is the policy read, is there a named owner per system, does incident response exist? Binary and easy to verify — see the one-page AI risk register.

4. Cancellation rate. Yes, a target for stopping things. A portfolio with no cancellations is unmanaged. This legitimises the hardest part of the job — how to kill an AI project.

5. Organisational capability. Can functions now evaluate a tool without the CAIO in the room? A role that makes itself progressively less necessary for routine decisions is working.

The tempting measures that misfire

Number of AI projects launched. Produces launches. Also produces a portfolio nobody can maintain.

Adoption percentage. Produces mandates, which destroy the signal — the argument in AI adoption metrics.

Cost savings alone. Produces aggressive claims against soft baselines. Savings are worth measuring, but only against a baseline recorded before the work.

Speed to deploy. Produces skipped evaluation and skipped governance, both of which surface later as incidents.

When to judge

Not in the first quarter. The first 90 days should produce an inventory, baselines, a governance minimum and at least one cancellation — a decision set, not results. Measuring return before there are measured baselines just rewards optimistic estimates. A Chief AI Officer's first 90 days sets out what to expect instead.

From quarter two onward, the five above are fair. For a fractional engagement the same measures apply, at the same cadence — the seat is smaller, not the accountability.

Frequently asked questions

What KPIs should a Chief AI Officer have? Portfolio return against stated baselines, retrospective decision quality, governance completeness, cancellation rate, and whether the organisation can now make routine AI decisions without them.

Why measure cancellations? Because a portfolio where nothing ever stops is unmanaged. An explicit expectation of stopping things legitimises the hardest and most valuable part of the role.

Why not measure projects launched? Because you will get launches — including ones that should have been declined — and a portfolio too large to maintain.

When should the role first be assessed? From the second quarter. The first is for inventory, baselines and governance; judging return before baselines exist rewards optimistic estimates.