Stability AI takes its open approach to text with StableLM

According to Stability AI, the company behind Stable Diffusion, StableLM is a new family of open language models. The launch starts with 3 billion and 7 billion parameter models and points to larger versions ranging from 15 billion to 65 billion parameters.

The development matters because it brings to text the formula that made the company famous: releasing models that developers, researchers and businesses can download, study and adapt. In a market shaped by closed services such as ChatGPT and GPT-4, StableLM offers an alternative with the model weights publicly available.

Smaller models, but usable on more systems

A language model is a system trained on enormous collections of text to predict the next word and, in doing so, write, summarize, answer questions or generate code. The number of parameters — the internal values a model adjusts during training — generally provides an indication of its capabilities, although it does not determine the final quality on its own.

The first StableLM Alpha models have 3 billion and 7 billion parameters. They are much smaller than the large commercial models whose size has not been disclosed, but that scale offers a practical advantage: it requires fewer resources to run the system and makes it easier for an organization to deploy it on its own infrastructure.

Stability AI says it trained these models on 1.5 trillion tokens, the units of text a model processes, which can correspond to complete words, parts of words or punctuation marks. The company explains that the training set draws on The Pile, a public collection of texts created for research, expanded with new experimental sources.

An open license with conditions

According to Stability AI, StableLM’s weights have been published on Hugging Face under the Creative Commons Attribution-ShareAlike 4.0 license. This allows the model to be used, modified and redistributed, including commercially, as long as the original authors are credited and derivative versions are shared under the same license.

That is not exactly the same as releasing the entire creation process. Publishing the weights makes it possible to run and fine-tune the model, but it does not necessarily allow anyone to reproduce its training from scratch: that would require sufficient data, code, configuration and computing power. Even so, it provides a much higher level of access than text assistants available only through a website or an API.

Alongside the base models, Stability AI has released instruction-tuned versions. This additional tuning teaches the system to behave like a conversational assistant rather than merely continue text. It is the step that turns a general model into a tool capable of responding to requests written in natural language.

The challenge is not just opening up the weights

The experience of Stable Diffusion shows why this strategy could have an impact. Making the model available fueled a developer community that built interfaces, extensions and specialized versions for visual styles, professional workflows and modest hardware. StableLM could attract a similar ecosystem for private chatbots, writing tools or internal assistants.

But language models present their own problems. They can invent facts with apparent confidence, reproduce biases in their training data or generate inappropriate content if safeguards are not put in place. Smaller versions will also face clear limitations compared with the best closed systems in reasoning, factual knowledge and following complex instructions.

Stability AI’s promise is to expand the family with larger models. Its success will depend on more than the parameter count: data quality, results on independent tests, ease of fine-tuning and clarity about the models’ limitations. For now, StableLM opens a new path for anyone who wants to experiment with text models without relying entirely on the major commercial platforms.