A local model can decide when to ask the cloud for help

On July 25, 2026, a preprint proposed a way to decide when a local model should answer and when a query should go to a stronger cloud model. CARGO measures agreement among several local responses: if they converge, the system may be more willing to run locally; if they diverge, it can offload the task. This is an uncertainty signal, not proof that the answer is true.

The 60-second decision

A local-cloud system has more than two absolute choices. It can reserve cloud capacity for difficult tasks while keeping others on-device or in private infrastructure. Before adopting one, ask: what signal triggers offloading, how many local runs are needed to calculate it, and what error is it meant to avoid? Without those answers, “cloud savings” can hide added latency or poorer outcomes.

The Routing Without Training preprint, arXiv:2607.20481v1, was submitted on May 30, 2026. Its authors call their training-free framework CARGO. It uses prompt-varied sampling, estimates agreement, applies Bayesian early stopping, and calibrates a target local-cloud collaboration ratio at deployment time.

What agreement measures

When local answers reach similar conclusions, it can be reasonable to treat the task as more stable. When they diverge, there is a reason to ask a stronger model. But matching answers can repeat the same mistake, and different answers can reflect an ambiguous question that needs human clarification rather than more compute.

The transferable skill is naming the signal accurately: internal agreement, not factual verification. For an answer that affects a purchase, diagnosis, or policy, evidence must still be a source, a checkable rule, or human review.

A test you can run

Evaluate four numbers together: cloud-offload rate, total cost, total latency, and final task quality. Compare with simple baselines: always offload, always answer locally, or route by query length. If a router improves only one metric while degrading the others, it has not solved the operational problem.

The authors report that CARGO outperforms training-free baselines and, in some settings, supervised routers. That is a result for their reasoning and question-answering tasks, models, and configurations; it is not proof of general production performance.

The durable lesson is that dividing work between models may not require training another model, but it does require measuring the cost of deciding. A good router is not one that sends the fewest queries to the cloud. It preserves the needed quality through a decision that can be inspected and corrected.