Federated learning: training models on distributed data

On 17 February 2016, a Google team led by H. Brendan McMahan published "Communication-Efficient Learning of Deep Networks from Decentralized Data," the paper that gave an idea its name. The authors put it this way: they advocate "an alternative that leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates. We term this decentralized approach Federated Learning."

The promise reduces to a sentence that has appeared in every industry deck since: your data never leaves your phone. And it is true. The trouble is that almost everyone hears something else — "your data is safe" — and those two statements are not the same. Telling them apart is the most useful thing you can take from this article.

How it works, in four steps

The central server sends the current model to a set of devices. Each one trains it a little on the data it already holds — your messages, your photos, your habits — and computes an update: the difference between the model it received and the one it ended up with. That update, not the data, travels back. The server averages updates from thousands of devices, obtains an improved model, and repeats the cycle.

The original motivation was not primarily privacy but logistics. The authors say so bluntly: "Communication costs are the principal constraint," and their method cuts the required rounds "by 10-100x" compared with synchronised stochastic gradient descent. Moving models is cheaper than moving petabytes of photos.

There is a second obstacle, subtler and very real. The data on each device are not a random sample of the world: a phone's photos belong to one person, in certain places, with certain tastes. In the jargon they are "unbalanced and non-IID," and the authors highlight that their method proved robust to that property, which they describe as a defining characteristic of this setting. An algorithm that averages wildly dissimilar contributions and still converges is not a detail: it is the reason this works outside the lab.

Where it is actually used

On 6 April 2017, McMahan and Daniel Ramage described the first real deployment: Gboard, the Android keyboard, to improve query suggestions. "When Gboard shows a suggested query," they explain, "your phone locally stores information about the current context and whether you clicked the suggestion."

The operational details matter more than they look, because they are what makes the idea viable on a device somebody carries in a pocket: "training happens only when the device is idle, plugged in, and on a free wireless connection, so there is no impact on the phone's performance."

And that same text, worth underlining, does not merely promise: it names the mechanism. Under secure aggregation, "a coordinating server can only decrypt the average update if 100s or 1000s of users have participated — no individual phone's update can be inspected before averaging." That is the right way to make a privacy claim: accompanied by the technique that backs it and by its threshold.

It is also worth separating two scenarios that share a name and are different problems. In cross-device federation, millions of unreliable phones take part, each holding very little data, and half drop out mid-round. In cross-silo federation — several hospitals, several banks — there is a handful of stable participants with enormous datasets each, and the obstacle stops being technical and becomes contractual and regulatory. Solutions do not transfer between the two.

"Does not leave" and "cannot be known" are different things

Here is the core. What leaves the device is not the data, but neither is it an innocuous number: it is a mathematical function computed from that data. And functions can sometimes be inverted.

In June 2019, Ligeng Zhu, Zhijian Liu and Song Han published a paper whose title admits no ambiguity: "Deep Leakage from Gradients." Their abstract opens by describing the belief they came to demolish: "For a long time, people believed that gradients are safe to share: i.e., the training data will not be leaked by gradient exchange. However, we show that it is possible to obtain the private training data from the publicly shared gradients."

And they are not describing a vague or statistical leak. Their results, in their words, show that "the recovery is pixel-wise accurate for images and token-wise matching for texts." That is: from the update a device sends, the specific image it trained on can be reconstructed. The authors close by saying they want "to raise people's awareness to rethink the gradient's safety."

What the field did about it

This needs saying plainly, because the alternative is alarmism: none of this invalidates federated learning, and the people building it have known for years. Defences exist, and they are real.

Secure aggregation means the server can only decrypt the sum of many updates, never any single participant's. Differential privacy adds calculated noise to each contribution, so that how much can be learned about an individual becomes mathematically provable. And the leakage paper itself offers its own remedy: the authors note that "the most effective defense method is gradient pruning" — sending only part of the update rather than all of it.

What matters, and what a brochure almost never says, is that none of the three is free. Differential privacy's noise costs accuracy. Secure aggregation costs computation and coordination. Pruning gradients costs learning speed. Privacy in these systems is not a property you either have or lack: it is a budget you spend, and somebody decides how much.

The capability you take away

Generalise the pattern, because you will meet it constantly and almost never under this label. Whenever somebody assures you they are not sending your data but something derived from it — aggregates, anonymised records, statistics, telemetry, "just metadata," a "hashed" identifier — the right question is not whether the original travels. It is this:

What can be reconstructed from whatever does travel?

Nearly always, more than it seems. A derivative retains information about its origin; that is precisely the point of a derivative, because if it retained nothing it would be useless for training anything. Usefulness and leakage are, up to a point, the same property seen from two sides.

And the second question, which separates a serious commitment from a marketing line: which specific defence was applied, and what did it cost? Someone who answers "secure aggregation at this parameter, and it costs us this much accuracy" is managing a risk. Someone who answers "your data never leaves your device" is answering a different question.

Where to go next, with no middlemen

Both papers are public and free. McMahan and colleagues' is surprisingly readable and lays out the averaging algorithm in detail. Zhu, Liu and Han's includes the reconstructed images, and seeing them changes your intuition about what a gradient is.

The capability you take from this is refusing to confuse where a piece of data sits with control over it — and always asking what can be rebuilt from whatever does get shared.