Number Research
← Back to Explore
Information Science Social Informatics Algorithmic Culture

The Map That Copies Our Prejudice by Design

Word2vec was rejected in 2013. Its 'king minus man plus woman' trick is quietly staged — and its bias is the same feature that makes it work.

S.J. Nam 7 min read
The Map That Copies Our Prejudice by Design

In early 2013, a short paper with an unglamorous title — "Efficient Estimation of Word Representations in Vector Space" — was rejected by the reviewers at ICLR, one of machine learning's marquee conferences. The lead author was Tomáš Mikolov, then at Google, and the idea inside would go on to reshape how nearly every AI system on earth handles language. It also took months just to get the code approved for release. The thing that now quietly powers your search results, your chatbot, and your song recommendations began life as a manuscript the experts didn't want.

The idea was called word2vec, and it did something that still sounds like a magic trick: it turned words into lists of numbers — coordinates — such that words used in similar ways ended up sitting near each other. That's an embedding. Strip away the jargon and it's the most important, least understood concept in modern AI. And the way it's usually explained gets the moral of the story backwards.

Meaning as a Neighborhood, Not a Definition

Here is the trap most explainers fall into. They say embeddings let a computer "understand" what words mean. That's a flattering story, and it's wrong in an instructive way.

An embedding doesn't know that a dog is a furry animal that barks. It knows that the word "dog" tends to show up near words like "leash," "vet," "bark," and "loyal," and rarely near "quarterly" or "titanium." That's the whole trick. The machine reads an enormous pile of text — the version trained on Google News chewed through roughly 100 billion words — and for each word it keeps a running tally of the company that word keeps. Words that keep similar company get placed close together on the map. Words that don't drift apart.

Think of it as a city with no street signs, only neighborhoods. "Coffee" and "espresso" live on the same block. "Berlin" and "Germany" are a short walk apart. "Walk" and "ran" are neighbors; so are "but" and "however." Nobody wrote down a definition anywhere. Location is the definition. The linguist J.R. Firth put the intuition in a line long before computers could act on it: you know a word by the company it keeps.

This is why calling it "understanding" is a category error, and why the correction matters. The machine is doing statistics on proximity, not comprehension. Once you see that clearly, both the genius and the danger of embeddings stop looking like separate topics. They're the same fact viewed from two angles.

The Party Trick That's Quietly Rigged

Every embeddings explainer eventually reaches for the same showstopper: take the coordinates for "king," subtract "man," add "woman," and you land almost exactly on "queen." Meaning, apparently, obeys arithmetic. Gender is just a direction you can travel on the map. It's a genuinely beautiful result, and it did more than any equation to convince people something profound was happening.

It's also a little bit staged, in a way the crowd is rarely told. When researchers run that calculation, the answer the machine actually computes as closest is usually "king" itself — the word you started with. To get the crowd-pleasing "queen," the standard procedure explicitly throws out the three input words ("king," "man," "woman") before picking the nearest neighbor. Exclude them by hand, and queen wins. Leave them in, and the honest result of king − man + woman is just... king.

I don't raise this to debunk embeddings. The relationships are real; the geometry does bend in the direction of gender, and that's remarkable. But the tidy demo has a stage magician's edit in it, and pretending otherwise sells the wrong lesson. Embeddings don't reason. They record. What looks like logic is the residue of statistical habit — millions of sentences in which kings and queens were discussed the same way, minus a nudge. The moment you accept that these vectors are a compressed photograph of how humans actually talk, you're ready for the uncomfortable part.

Smart and Prejudiced Are the Same Setting

In 2016, a team led by Tolga Bolukbasi, with collaborators including researchers at Microsoft and Boston University, ran the arithmetic trick on a different pair. They asked the standard embeddings: man is to computer programmer as woman is to what? The system answered "homemaker." They published the finding under exactly that title — "Man is to Computer Programmer as Woman is to Homemaker?" — and showed that gender bias wasn't scattered noise. It sat along a clean, measurable direction in the space, the same kind of direction that gives you king-to-queen.

This is the observation I wish every beginner met on day one, because it dissolves a comfortable myth. The bias is not a defect someone forgot to remove. It is the identical mechanism that makes embeddings useful, running exactly as designed. If your model learned that "Berlin" and "Germany" belong together because people write them together, it learned that "programmer" and "he" belong together for the very same reason — because, in the text we produced, they did. You cannot have the machine that captures "Paris is to France as Tokyo is to Japan" and also demand a machine innocent of the culture that wrote those sentences. It's one machine. It reflects us, flatteringly and unflatteringly, with no way to tell the two apart on its own.

That's the counterintuitive core of the whole subject: the intelligence and the prejudice are not a feature and a bug. They're a single capability pointed at a mirror.

Why This Sits Under Almost Everything Now

You might expect a decade-old idea about word lists to have been superseded. Instead it got promoted. Embeddings graduated from single words to whole sentences, paragraphs, images, and audio, and they became the plumbing of the current AI boom.

When a chatbot answers a question about your company's internal documents, it usually isn't reading all of them. It's converting your question into an embedding, then searching a vector database for the stored chunks whose coordinates sit closest to your question's — the same "who are your neighbors" logic, at industrial scale. That pattern, retrieval-augmented generation, is how many systems stay current and cut down on confident nonsense: fetch the nearby facts first, then let the language model talk. Spotify represents songs, artists, and listeners as embeddings so it can measure how close your taste sits to a track you've never heard. Modern search engines match intent rather than keywords for the same reason. Proximity became the master interface.

So the humble insight from that rejected 2013 paper — that you can pin meaning to a location and let closeness do the reasoning — turned out to be less a technique than an operating principle. It's under the search bar, the recommendation feed, the assistant, the fraud filter.

Which leaves a sharper question than "how do embeddings work." We've built the machinery of the internet on a device whose entire talent is faithfully absorbing the patterns in human text — including the ones we'd disown if asked directly. The engineers cleaning gender out of "homemaker" are not fixing a broken tool. They're arguing with a mirror about what it's allowed to show. The map was never neutral, because the territory it copied never was. The interesting fight of the next decade isn't teaching machines to understand us. It's deciding which parts of ourselves we want them to keep quoting back.

References