Stability AI takes its open approach to text with StableLM

On April 19, 2023, Stability AI released the first StableLM Alpha models; the launch helps distinguish accessible weights, licensing, data and reproducibility.

According to Stability AI, the company behind Stable Diffusion, StableLM is a new family of open language models. The launch starts with 3 billion and 7 billion parameter models and points to larger versions ranging from 15 billion to 65 billion parameters. Primary source

The development matters because it brings to text the formula that made the company famous: releasing models that developers, researchers and businesses can download, study and adapt. In a market shaped by closed services such as ChatGPT and GPT-4, StableLM offers an alternative with the model weights publicly available.

Smaller models, but usable on more systems

A language model is a system trained on enormous collections of text to predict the next word and, in doing so, write, summarize, answer questions or generate code. The number of parameters — the internal values a model adjusts during training — generally provides an indication of its capabilities, although it does not determine the final quality on its own.

The first StableLM Alpha models have 3 billion and 7 billion parameters. They are much smaller than the large commercial models whose size has not been disclosed, but that scale offers a practical advantage: it requires fewer resources to run the system and makes it easier for an organization to deploy it on its own infrastructure. Primary source

Stability AI says it trained these models on 1.5 trillion tokens, the units of text a model processes, which can correspond to complete words, parts of words or punctuation marks. The company explains that the training set draws on The Pile, a public collection of texts created for research, expanded with new experimental sources. Primary source

An open license with conditions

According to Stability AI, StableLM’s weights have been published on Hugging Face under the Creative Commons Attribution-ShareAlike 4.0 license. This allows the model to be used, modified and redistributed, including commercially, as long as the original authors are credited and derivative versions are shared under the same license. Primary source

That is not exactly the same as releasing the entire creation process. Publishing the weights makes it possible to run and fine-tune the model, but it does not necessarily allow anyone to reproduce its training from scratch: that would require sufficient data, code, configuration and computing power. Even so, it provides a much higher level of access than text assistants available only through a website or an API.

Alongside the base models, Stability AI has released instruction-tuned versions. This additional tuning teaches the system to behave like a conversational assistant rather than merely continue text. It is the step that turns a general model into a tool capable of responding to requests written in natural language.

The challenge is not just opening up the weights

The experience of Stable Diffusion shows why this strategy could have an impact. Making the model available fueled a developer community that built interfaces, extensions and specialized versions for visual styles, professional workflows and modest hardware. StableLM could attract a similar ecosystem for private chatbots, writing tools or internal assistants.

But language models present their own problems. They can invent facts with apparent confidence, reproduce biases in their training data or generate inappropriate content if safeguards are not put in place. Smaller versions will also face clear limitations compared with the best closed systems in reasoning, factual knowledge and following complex instructions.

Stability AI’s promise is to expand the family with larger models. Its success will depend on more than the parameter count: data quality, results on independent tests, ease of fine-tuning and clarity about the models’ limitations. For now, StableLM opens a new path for anyone who wants to experiment with text models without relying entirely on the major commercial platforms.

“Open” has several testable layers

Weights make it possible to run and modify a model, but do not automatically reconstruct how it was created. Assess openness by asking separately whether weights, inference code, training code, data, preparation recipe, evaluations and licence are available. StableLM Alpha exposed base models, while the launch post distinguished those bases from research instruction-tuned models under a different licence.

That distinction changes what a user may do. A licence can allow commercial use of the base model while restricting a conversational variant. Fine-tuning datasets also carry their own terms. Before integrating a model, do not rely on a label on a webpage: locate the licence file for the exact version and record its hash or identifier.

Parameter count measures architectural size, not task quality. A well-trained smaller model may beat a larger one in one domain and fail at general reasoning. Fair comparison keeps prompt, quantisation, hardware and test set constant. If several variables change, attributing the result to size is speculation.

Reproducibility includes data

Stability AI described an experimental dataset based on The Pile and expanded threefold to 1.5 trillion tokens, but said details would come later. Preserve that limitation when describing the release: volume alone does not expose duplication, provenance, languages, personal data or protected material. Primary source

Local deployment gives an organisation control over where inputs are processed, but transfers security work to that organisation. It must inspect dependencies, restrict access, test prompt injection, record versions and decide how updates occur. “On our servers” reduces certain data transfers; it does not make a system private or secure by itself.

A minimum evaluation combines quality, cost and risk. Choose real tasks, measure accuracy, memory, latency and harmful failures, then repeat after each adjustment. Include a closed model as a reference when the actual decision is between the two, rather than treating it as an abstract enemy.

The transferable skill is to replace the word “open” with a seven-field record: weights, code, data, recipe, evaluation, licence and change governance. Mark missing items as limits. This lets different projects be compared without rewarding a label that every provider defines differently.

Hardware must be documented because it changes the experience. Quantisation reduces memory at the cost of possible quality loss; different libraries produce different speeds; a larger context window consumes more resources. Saying only “runs locally” hides whether the test required a professional GPU or fits on an ordinary laptop. A table with peak memory, tokens per second and configuration turns accessibility into a repeatable claim.

Preserve the base model alongside every adaptation. If a specialised version changes behaviour, the team needs to know which data were added, which licence it inherits and how to roll back. That traceability separates a weakness in the original model from a failure introduced during fine-tuning.

Before downloading, identify who maintains the repository and how vulnerabilities are communicated. Weights may remain available while the code loading them changes. Signatures, hashes and an isolated environment reduce the risk of confusing a known artefact with a later substitution. Opening a model expands inspection capacity; using it requires a documented supply chain and a named person responsible for reviewing every update.

Without that owner, the ability to inspect may remain a theoretical advantage.