Databricks launches DBRX, a 132B open model

Databricks has unveiled DBRX, an open language model designed to compete with the best publicly available models. Its main strength is combining a scale of 132 billion parameters with a system that uses only part of them for each response, a design intended to improve performance without sending usage costs soaring.

DBRX arrives as open models—those whose weights can be downloaded and run outside their creator’s infrastructure—have become an increasingly serious alternative to the closed APIs offered by OpenAI, Google, and Anthropic. For businesses, that distinction matters: it lets them adapt the model to their own data and decide where processing takes place.

A large model that doesn’t activate its full capacity

DBRX uses a mixture-of-experts architecture, or MoE. Rather than using all its parameters for every piece of text, the model contains 16 specialized networks—the experts—and activates four of them to process each token, the smallest unit of text handled by a language model.

The result is 132 billion parameters in total, but 36 billion active at each generation step. Parameters are the values a model adjusts during training to learn patterns in language, code, or mathematics. Activating fewer of them does not necessarily mean knowing less: it allows the model to reserve capacity for different tasks and reduce the computation required for each response.

The technique is not new. Google has used it in some of its research, and Mistral popularized it among open models with Mixtral 8x7B. DBRX’s difference lies in its scale and the number of experts selected per token. Databricks says the model was trained on 12 trillion tokens and supports a 32,000-token context window, enough to analyze lengthy documents or sustain long conversations.

Better results than leading open models

The company has released two variants: DBRX Base, aimed at developers and researchers who want to fine-tune it for a specific task, and DBRX Instruct, designed to follow instructions in a conversational format.

According to results published by Databricks, DBRX Instruct outperforms open models such as Llama 2 70B, Mixtral Instruct, and Grok-1 on standard tests of language understanding, programming, and mathematical reasoning. On HumanEval, a Python code-generation test, it scores 70.1%; on MMLU, a battery of academic and professional questions, it achieves 73.7%.

Benchmarks should be interpreted with caution. They are useful for comparing models on controlled tasks, but they do not guarantee that one will perform better in a real-world enterprise setting, where data quality, tool integration, security, and the cost of serving millions of queries all matter. Even so, the figures place DBRX among the most capable open models available today.

Conditional openness with an enterprise focus

Databricks is distributing DBRX’s weights under its own license, the Databricks Open Model License, and has made them available through Hugging Face. The license allows the model to be used, modified, and distributed, including for commercial purposes, although it is not a standard open-source software license such as Apache 2.0. Companies should review its terms before incorporating the model into a product.

The launch fits Databricks’ business, which focuses on enterprise data and machine-learning platforms. An open model can run on a customer’s infrastructure, be fine-tuned with internal information, and connect to its analytics systems. But running a model of this size still requires substantial infrastructure, even if the mixture-of-experts architecture reduces computation compared with an equivalent dense model.

DBRX does not erase the advantage held by large closed models, which tend to offer more polished products, integrated tools, and managed operations. It does narrow the gap. For organizations that need greater control over their data or want to avoid relying entirely on a single API, the new model expands the available options.