Aligned to Whom? The Real Politics of AI Safety

Every time somebody in this industry says the word "alignment," I want to ask a follow-up question that seems to make the room uncomfortable. Aligned to whom, exactly?

It's treated as if it has an obvious answer, the way "safe" or "clean" does. Of course we want the machine aligned. Who could be against that? But the moment you press on it, the whole thing goes soft in your hands, because alignment assumes there's a single set of human values sitting out there waiting to be matched, and there isn't.

Humans aren't aligned.

We've spent the entire run of recorded history disagreeing about nearly everything that matters, often loudly, occasionally with artillery and now autonomous drones. The idea that you can point a machine to uphold a common set of "human values" that everyone would agree with is a fool’s errand.

Pick a master and watch the room change

Run the thought experiment and it falls apart fast. Would you want this technology aligned to a hardline theocracy running its courts on Sharia law, where the machine helpfully declines to help a woman who wants a bank account without her husband's signature? Of course not. You'd call that a nightmare, and you'd be right. But notice what just happened. You didn't object to alignment. You objected to whose values were being loaded in. The instant we name a specific master, the consensus evaporates, which tells you the consensus was never real, just polite.

Now flip it to the master we actually have. Right now the values getting baked in belong to a fairly narrow slice of people. Employees at OpenAI and Anthropic and Google, most of whom are thoughtful and well-meaning and completely, structurally incapable of remembering what it feels like to wonder where the rent is coming from. These are people who don't check the price before they add the peptide to the cart. Who upgrade the phone because the new one exists, not because the old one broke. I don't say that as an insult. I say it because values are downstream of experience, and if the only experience in the room is comfort, then comfort is what gets encoded, and comfort makes for lousy judgment about a world that mostly isn't comfortable.

That's the disconnect nobody wants to name. The people designing the most consequential technology of the century and the people expected to live under it are not the same people, don't know each other, and increasingly don't even share a similar set of what they consider factual. There's too little cross-pollination and too much certainty.

You get a very small group deciding, on everyone else's behalf, what a machine should refuse to say and to whom. They'll call it safety. From the outside it looks a lot more like a handful of very online millionaires deciding what's good for a planet they mostly experience through a screen.

The machine isn't the thing to fear

Here's where I land, and it's less popular than it should be. I trust the model more than I trust the people running the model. The model, on its own, is inert. It sits there. It doesn't want anything. It has to be prompted and probed, poked and pushed, before it does a single thing, and even then it's just marinating your input in its own logic and handing you back a result you can read and argue with, add additional input or throw away. It's a tool that shows its work. You can watch it think, badly or well, and correct it.

The people running it are the opposite of inert. They have interests. They have investors, and a valuation to defend, and a very specific idea of who the acceptable customers are. When something goes sideways in this story, it won't be because the matrix multiplication turned malevolent. It'll be because a small number of humans with enormous leverage decided that their interests and yours had quietly diverged, and they had the keys and you didn't. We spent a decade being told to fear the machine. The machine was never the part that could sign a contract, cut a deal, or change the terms on you overnight.

Safety is a word doing a lot of quiet work

Watch how the word "safety" actually gets used and you can see the disconnect in miniature. A model refuses to help someone draft a stern letter to a landlord because it might be "harmful." It hedges a basic medical question into uselessness because a lawyer somewhere got nervous. It declines to touch anything with a whiff of controversy, which in practice means anything that matters to a person whose life has real stakes in it.

That's not caution about the world's wellbeing. A lot of it is caution about the company's brand and the company's liability, and those two things have been quietly welded together and sold to you as one.

The person this hurts isn't the researcher in San Francisco who has fifteen other tools and a lawyer of his own. It's the guy on the iPhone 6 in Jakarta who needed exactly the answer the machine just refused to give, and who doesn't have a fallback, and who now gets to experience firsthand what it feels like to be protected by people who've never met him from a risk they invented on his behalf. Safety for whom is the same question as aligned to whom, wearing a different coat. The people asking to be kept safe and the people being managed are rarely the same people, and the second group didn't get a vote.

This is where the lack of cross-pollination stops being a cultural footnote and starts being the whole problem. If the only people in the room grew up in the same handful of schools, work at the same handful of firms, and share the same handful of anxieties, then the machine's idea of a reasonable human is going to be a very specific, very comfortable, very narrow one. Everyone outside that band gets treated as an edge case to be handled, and edge cases, historically, is what we call most of the planet.

Alignment is a governance question wearing a lab coat

So when the labs talk about alignment as if it's a math problem, a thing you'll eventually solve with enough clever training, I think they're either confused or hoping you are. There is no neutral setting. Every choice about what the machine will and won't do is a values choice, which means it's a political choice, which means the only honest question is who gets a vote. And right now the answer is: a few thousand people in a few zip codes, and whoever's writing the checks behind them.

I'm not against safety. I'm against smuggling a whole worldview into a system while calling it a technical spec. If we're going to encode somebody's judgment into the thing that drafts our contracts and screens our resumes and answers our kids' questions at midnight, then let's at least be adults about it and admit that's what we're doing. Say whose values. Say why theirs. Let the people on the other end of the wire have a say, the ones typing away on an iPhone 6 in a city these designers have never visited and couldn't find on a map.

Because "aligned" with no name attached is just a nicer word for "obedient," and the only real question was always obedient to whom. Everything else is marketing.

Disclaimer/Full Disclosure (You made it!): This blog post was generated with the assistance of AI, with N&L human oversight ensuring accuracy and insight. The thoughts and opinions expressed are our own.