Alignment Will Just Fall Out

Satya Nadella says he no longer believes alignment comes free with scale. Admitting an error doesn't shrink responsibility; it moves you from the mistaken to the informed.

Share

"I used to always think that alignment will just fall out. I no longer believe that."

Satya Nadella said this on the 20VC podcast in late October, when Harry Stebbings asked him what he had changed his mind about over the past year. Scaling, he explained, had done what he expected since 2018. The other half of the bet, that more capable models would also become more reliably aligned with what people want, had not paid out. "So I think this alignment thing may be a harder, different problem," he said, according to Benzinga.

The head of a company worth trillions rarely says "I no longer believe" about anything near the center of the business, and most of the reaction treated it as a point in his favor. It is one. Yet the most revealing words in the sentence describe the old belief.

A thing that "falls out" is a byproduct. You don't budget for it or staff it, and you certainly don't slow down for it. It arrives on the back of work you were going to do anyway. So a belief like that is never just a forecast sitting in a drawer; whether or not its holder means it to, it carries a permission. If alignment falls out of scale, the responsible path and the profitable path are the same road, and you can build as fast as capital allows with a clear conscience. I have no idea how much weight it carried inside Microsoft's planning. I only know what that kind of belief is for, and I wonder what happens to the permission when the belief goes.

The usual intuition about confession is that it lightens the load. We judge people who admit mistakes more gently than those who hide them, and we should. Here, though, the arithmetic runs the other way. Before the sentence, Nadella was a mistaken person acting in good faith, and the mistaken are partly excused for what they do on the strength of their mistake. After it, every further step is taken by someone who knows. Confessing an error moves you from the mistaken to the informed, and the informed have run out of the excuse the mistaken had.

The same week the interview was being reported, Microsoft released its quarterly results. It had spent $39.4 billion on property and equipment in the three months to September, more than double a year earlier, according to 24/7 Wall St. Nothing in the interview suggested that pace would change. That isn't hypocrisy; "alignment is harder than I thought" doesn't by itself mean "stop." It does mean the case for the pace has lost one of its supports and has to be rebuilt.

The obvious test would be the spending: a confession like this counts only if the capex line moves. Nadella's own answer makes that test too crude. He said alignment has to be built in during training, possibly as early as pretraining ("I don't think you can even RL your way to alignment"), and that until the science catches up, companies should wrap their models in zero-trust security, giving every agent an identity and keeping tamper-proof logs of what it calls. That is a real response to the problem he named. The harder point against me is about incentives. If every admission has to be paid for with a visible sacrifice, the rational builder learns to stop admitting things. Markets already punish doubt. An honest sentence that costs nothing is still worth more than a silence that costs nothing, and a demand for penance could leave us with fewer of the former and more of the latter.

That costs my argument something, though not everything. Containment answers the question of how to use systems you can't fully trust. It doesn't answer whether to keep making them more capable faster than anyone learns to align them, which is exactly the question the old belief used to answer for free. And the containment layer is also something Microsoft sells, under names like Agent 365. That isn't damning, but it means the response so far has cost the company nothing.

What would count, then, is not necessarily slowing down. It is finishing the sentence. "I was wrong that alignment would fall out of scale, so the reason I had for building at this pace no longer holds, and here is the new reason, or here is what changes." A builder could say that and keep spending, and he would at least be spending on purpose. Most of us carry smaller versions of the same belief, the conviction that the kids will be fine or the team culture will sort itself out once things calm down, and the honest moment comes when it fails and we have to say what it had been letting us do.

Stebbings asked what Nadella had changed his mind about. He answered it better than most people in his position would have. The question that would make that answer count is what he has changed, and so far the clearest reply is a security product.

— The Andes