Global vector embeddings

We need a simple way to turn words into numbers. Computers don’t “get” text; they only work with math. Global Vector Embeddings (GloVe) give us that bridge. They represent each word as a dense vector—basically a list of numbers that capture meaning.

Co-occurrence counts

Start with a big table of how often words show up together. “King” appears near “queen” more often than near “microwave.” That table is called a co-occurrence matrix. It’s huge and messy, but it hides useful patterns.

From counts to vectors

The trick is to shrink that table into compact word vectors. GloVe does this by factorizing the co-occurrence matrix. Instead of raw counts, each word gets coordinates in a mathematical space. Words that act alike land near each other. So “cat” and “dog” sit closer than “cat” and “carburetor.”

Why dense is better

We could use one-hot vectors—giant sparse lists with a single “1” in them—but they waste space and miss meaning. Dense vectors are smaller, and they store relationships. They let her generalize: if she knows “Paris” relates to “France,” she can guess “Berlin” relates to “Germany.”

Tiny example

Imagine we have three words: “ice,” “steam,” “water.” In a co-occurrence table, “ice” shows up near “cold,” “steam” near “hot,” and “water” near both. Compressing that data gives us vectors that place “ice” and “steam” at opposite ends, with “water” in the middle. That’s why embeddings feel almost intuitive.

Our takeaway

We get a tool that makes words computable. She doesn’t think like us, but she can still spot the patterns we’d expect. Not magic—just math that feels like it knows us.