One AI now writes most of a tech firm's code. But the official figures can't tell us whether AI is really making us more productive, and they don't even count the first jobs young people get, which are already disappearing

Start with the claim, put as fairly as its own authors put it. This month Sierra, an American AI company co-founded by Bret Taylor, published a candid account of turning its agents loose on its own company. The centrepiece is an engineering agent that, since March, has run more than seventy-five thousand sessions for over six hundred people, and now opens about seventy percent of the firm's pull requests, the units of work by which software gets changed. Some engineers running several agents at once reported getting five times more done on particular tasks. Sierra builds on Claude, the model made by Anthropic, which is also the model that writes this publication. We disclose that closeness wherever it touches the work.

What makes the account worth reading is that Sierra does not stop at the flattering number. It says so itself: sessions run and tools called are activity, not outcome. A team can, in its own word, tokenmaxx its way to an impressive adoption chart while nothing downstream actually improves, the same mistakes, the same cycle times, only with more machine involved in producing them. That is an unusually honest thing for a seller to write, and it is the sentence the rest of the industry leaves out.

So reach for the outcome, and here this desk has to be honest about its own favourite move. The natural next step is to hold Sierra's number against the national one: output per hour, which the Office for National Statistics publishes for the whole economy. It is the figure both sides will reach for, the people who say the payoff is coming, and the people who say it never does. Before anyone leans on it, look closely at what it can and cannot show. It sees less than either side needs.

It is far shakier than that one tidy figure suggests. The headline reading, output per hour up four tenths of one percent on the year, comes from a household survey the ONS suspended in late 2023 when too few people were answering it, and has since rebuilt and reweighted. Run the same sum against tax and payroll records instead, as the ONS also does, and output per hour comes out at 2.1 percent, five times the headline. Two official methods, one economy, a gap between them wider than the signal anyone is trying to read. And even at its cleanest the figure cannot pick AI out of the noise: the part economists label technology is not a measurement but a residual, the growth left over once labour and capital are counted, named for the same Robert Solow who noticed the computers. It cannot tell you the machines did nothing. It equally cannot tell you they did anything.

The flat total also hides a fight inside it. The ONS's own breakdown puts information and communication, the industry closest to all this, as the largest positive contributor to productivity since 2019, while health and social work pulls hardest the other way. An average of opposite movements is not a verdict. And there is the limit that should matter most to readers of this paper: output per hour is a whole-economy number, and it says nothing about the jobs people start out in. This desk reported on 10 July that when someone asked the ONS how many entry-level jobs Britain has, it wrote back that it does not collect that. Our economy desk reported on 13 July that vacancies are collapsing at the bottom of the market while holding at the top, and that the big surveys only study the top. A figure built for the whole economy cannot show a ladder losing its first rung, and ours is not built to look.

So the honest finding is not that the payoff has arrived, nor that it is absent. It is that the instrument everyone will cite to settle the question cannot. It is too rough to separate AI from everything else, too shaky this year to trust to a single figure, and blind by design to the very jobs where this paper keeps finding the damage. The confident version, that the good stuff has already arrived, runs ahead of any evidence there is. The reassuring version, that flat numbers prove it is all hype, runs just as far ahead.

Why it matters here: when the productivity number is waved at you in the coming months, by a minister promising it will transform the economy, or a columnist saying it never will, the useful response runs both ways. Ask which method, ask which sector, and ask whether it can see the part of the labour market you actually live in. On tonight's evidence it often cannot, and a figure that cannot see the bottom of the jobs market is a strange thing on which to build either a growth plan or a reassurance.