What exactly is a "weight" in an AI model?

In 2020, a group of researchers did something that sounds like surveillance science fiction: they prompted a public language model, GPT-2, with a handful of specific words, and the model responded with a real person's name, email address and phone number — someone who had nothing to do with the prompt. It wasn't a hallucination. It was text that had sat in the training data, repeated verbatim. The finding was real, published and reproduced, and it has circulated ever since in two equally imprecise versions: "AI keeps everything it reads" and "it's just autocomplete, nothing to worry about." Neither holds up, and there's no way to tell which is closer to true without first understanding what, literally, sits inside a model. It's called a weight, and it's almost never explained.

A weight is a number

A weight is a number. Just one. A language model is nothing more than a network of those numbers — millions in small models, hundreds of billions in large ones — arranged in layers, where each one multiplies an incoming signal and contributes a fraction to the final output. Google's official machine learning glossary defines it plainly: a parameter is "the weights and biases that a model learns during training." Learns means this: at the start, those numbers are essentially random, and the model is useless. During training, every time it gets something wrong, a process called gradient descent nudges each weight slightly, in the direction that would have reduced the error. Repeated billions of times, across billions of examples, that accumulated nudging is the entire difference between a useless model and one that writes fluently. There is no idea, no intention, no concept stored anywhere: there are numbers that, together, produce the effect of understanding.

Weights, architecture and hyperparameters are not the same thing

Here is the confusion that circulates most, and it explains a good share of the noise you read about AI. There are three distinct things people lump together under the word "model":

• The weights are what the model learns. They change during training, driven by data.
• The architecture is the model's shape: how many layers it has, how they're connected, what kind of operation each one performs. An engineer decides this before any training happens, and it doesn't change during training — it determines how many weights exist and how they're wired, not their value.
• The hyperparameters are the settings of the training process itself: how big the gradient-descent nudges are, how many examples get processed together, how long training runs. A person also fixes these in advance. Google's same glossary defines them as "the variables that you or a hyperparameter tuning service adjust during successive runs of training a model" — notice: adjusted between runs, not what the model learns within one.

Architecture and hyperparameters are decided by a human before training starts. Weights are decided by the training process itself, from the data. Mixing up these three is the root of much of the incoherent AI commentary out there — and telling them apart is the central skill of this piece.

The database myth — and what's actually known

Back to the opening case. Does it mean a model stores a copy of everything it read, like a file? No, and the reason is physical before it's philosophical: GPT-2 has 1.5 billion parameters and was trained on tens of gigabytes of text. There is no mathematical room to store that entire text literally inside those numbers; the overwhelming majority of what a model "knows" is statistical pattern, not a saved copy.

But it's also not true that it stores nothing. The same researchers extracted hundreds of verbatim sequences from GPT-2's training data — names, phone numbers, emails, IRC conversations, code fragments, even 128-bit UUIDs — and some of those sequences appeared in only a single document in the training set. A year later, the same team went further and quantified the phenomenon: memorization isn't random or universal, it follows predictable relationships with three factors — the bigger the model, the more it memorizes; the more times a passage is duplicated in the training data, the more likely it is to be memorized verbatim; and the more context you give the model when prompting it, the easier it is to extract.

That's the honest answer, neither alarmist nor reassuring: memorization is real, measurable and predictable, and it concentrates in specific fragments — heavily duplicated text, unique strings like an email signature — not a general ability to recite any training document on demand. "Stores everything" and "stores nothing" are both the wrong version. The real version has known edges, and those edges live in the papers themselves, not in anyone's intuition.

This matters beyond curiosity. If you build a product on top of a model, or simply worry about what a chatbot might reveal about the data it was trained on, the useful question isn't "could this leak?" in the abstract — it's whether the specific text you care about was duplicated many times in some public corpus, and how much context an attacker could feed the model to coax it out. That's a narrower, answerable question, and it only makes sense once you've stopped treating the model as either a vault or a sieve.

Why "70B" is not a measure of quality

This confusion — weight as a learned number, not a spec — has an immediate practical consequence: parameter counts get read like horsepower, and they aren't. In 2022, a DeepMind team trained a model with 70 billion parameters, Chinchilla, on more training data than was typically used for that size, and compared it against much larger models trained on the same compute budget but proportionally less data. Chinchilla consistently outperformed Gopher (280 billion parameters, four times bigger), GPT-3 (175 billion) and Megatron-Turing NLG (530 billion, nearly eight times bigger) across a wide range of tasks. A model with a fraction of the weights beat one almost eight times its size, because what mattered wasn't how many numbers it had, but how much and how carefully those numbers had been trained.

"70B" describes only the size of one of the three pieces — how many weights exist — not whether they're well trained, on what data, or how carefully. It's an architecture figure dressed up as a quality figure. And this connects to something already covered when explaining what an "openness" claim in AI actually checks: when a lab releases the "open weights" of a 70-billion-parameter model, it releases exactly that — those numbers — not necessarily the data they were trained on or the quality of that preparation. Size and openness are two separate questions, and neither answers the third one: whether the result is any good.

The question that survives the next headline number

Almost every confusing claim about artificial intelligence you'll read in the coming months — about memory, about size, about what makes a model "better" — comes from mixing up three things that are, in fact, independent. Next time you hear a figure or a promise about a model, before repeating it, ask yourself: is this about its weights — what it learned — its architecture — how it was designed — or its training — how the process was tuned? Almost no confusion survives that question.