Weights

A weight is a number that an artificial intelligence model adjusts during training to reduce its errors. It's the smallest unit of what a model "learns": it multiplies an incoming signal and contributes, together with anywhere from millions to hundreds of billions of identical numbers depending on the model's size, to the final output. Google's official machine learning glossary defines a parameter — the technical term covering weights and biases — as "the weights and biases that a model learns during training."

At the start of training, a model's weights are essentially random, and the model produces nonsense. A process called gradient descent adjusts them repeatedly, nudging each number slightly in the direction that would have reduced the error on each training example. After billions of these small adjustments, the set of weights produces coherent, useful answers. No concept, idea or fact is stored as such in any single place inside the model: there are only numbers that, combined, produce the effect of understanding.

Weights should not be confused with a model's architecture (its shape: how many layers it has, how they connect to one another, decided by an engineer before training begins) or its hyperparameters (the settings of the training process itself, such as the learning rate or how many examples are processed at once, also fixed in advance by a person). The architecture and hyperparameters are decided by a human before training starts and stay fixed; the weights are the only piece the training process itself produces, from the data.

When a lab announces it is releasing a model's "open weights," it is releasing exactly these numbers for anyone to download and run — not necessarily the code used to train them or the original training data. And a figure like "70B" only tells you how many weights a model has, not whether they were trained well or on what: a model with far fewer weights but a better training recipe can outperform one many times its size.