Federated Learning

Federated learning trains a shared model without gathering the data: each client computes an update from what it holds locally, and a central server coordinates the aggregation of those updates. The term was introduced by H. Brendan McMahan and colleagues at Google in 2016, and the field's reference survey —signed by Peter Kairouz, McMahan himself and several dozen researchers— defines it with a sentence worth reading slowly: it embodies the principles of focused collection and data minimisation, and can mitigate many of the systemic privacy risks of centralised machine learning. Mitigate, not eliminate. That is where everything interesting begins.

The cycle, step by step

A round of federated training has five phases, and knowing them changes the conversation. The server selects clients from those meeting eligibility requirements —a phone typically qualifies only if it is plugged in, on an unmetered wi-fi connection and idle—. It broadcasts the current model and a training program. Each device computes its update on its own data. The server aggregates what comes back, dropping stragglers once enough devices have reported. And with that aggregate it updates the shared model, which goes out again next round.

Aggregation is the most important phase and the worst explained. It is not an engineering detail: it is the point where the techniques that make federation more than distributed computation are integrated — secure aggregation, lossy compression to save communication, and noise addition and update clipping when differential privacy is wanted. None of that comes by default: they are parts you have to fit, and each one costs.

What the original algorithm contributed

The paper that founded the field —presented at the AISTATS conference in 2017— proposed federated averaging and measured two concrete things. First: the method holds up under unbalanced and non-identically distributed data, which its authors describe as the defining characteristic of this setting; every person's phone writes different things, and that is not noise to be corrected but the starting situation. Second: the bottleneck is communication, and their approach cuts the required rounds by ten to a hundred times compared with synchronised stochastic gradient descent. That is its measured contribution, and it says nothing about privacy.

What size are we talking about

The Kairouz survey gives the orders of magnitude of a typical mobile deployment, and they are useful because they dismantle the picture of «a few devices collaborating». The total population runs from one million to ten billion devices. Each round selects between 50 and 5,000. Across the training of a single model, between one hundred thousand and ten million take part. Convergence needs between 500 and 10,000 rounds, and the clock reads one to ten days.

With those numbers come constraints that do not exist in a datacentre. Clients are unreliable: 5% or more of those selected in a round are expected to fail or drop out, because the phone stops charging, loses wi-fi or its owner picks it up. They are stateless: a device will typically take part just once in the whole task, so every round works with fresh participants. And they are not addressable: the system cannot call a specific client. All of this forces the training to be designed so that nobody notices anything — the model downloaded during a round is ephemeral and is not used to answer the user, precisely because a misconfigured training run produces bad predictions.

Two worlds under one name

The same term covers two situations best not conflated. In cross-device federation the clients are millions of phones or connected gadgets, communication is usually the bottleneck, and the data split is fixed by who generates what. In cross-silo federation the clients are between two and a hundred entities —hospitals, banks, datacentres of one company— that do have identities, do take part in every round, and can split data by examples or by features. What both share is the essential part: data stays decentralised and an orchestration server organises the training without ever seeing the raw records.

This is not theory: the survey documents that Google uses federation in the Gboard keyboard and in features of its phones and of Android Messages, that Apple has used it since iOS 13 in applications such as the predictive keyboard and the «Hey Siri» vocal classifier, and that there are deployments in medical research and hotword detection. It also lists the other world: reinsurance risk prediction, pharmaceutical discovery, electronic health record mining, medical image segmentation and smart manufacturing.

Why «the data doesn't leave» does not mean «it's private»

This is the inference to break, and not with opinions but with two published results.

The first looks inward. Deep Leakage from Gradients, by Ligeng Zhu, Zhijian Liu and Song Han, starts from a widespread belief —sharing gradients is safe, training data does not leak— and shows that private training data can be obtained from publicly shared gradients. In their experiments the recovery is pixel-wise accurate for images and token-wise matching for texts. The authors themselves discuss defence strategies; the message is not that every gradient leaks, but that sharing gradients can no longer be assumed harmless without stating under what conditions.

The second looks outward. How To Backdoor Federated Learning, by Eugene Bagdasaryan and colleagues, demonstrates that any participant can introduce hidden functionality into the global model: making an image classifier assign an attacker-chosen label to images with a certain feature, or a word predictor complete certain sentences with an attacker-chosen word. The backdoor enters through the same door legitimate learning does: an update sent by a client.

Neither experiment proves that every deployment leaks or ends up poisoned. What they invalidate is the automatic leap: «the data never leaves the device, therefore the system is private and trustworthy». Between those two statements sits a threat model that somebody has to write down.

The capability the reader takes away

Faced with any system presented as federated —and there will be more and more— five questions bring the conversation back to checkable ground, and none of them requires being a cryptographer.

Cross-device or cross-silo? It changes everything: the scale, who can be malicious, and which guarantees are realistic. What exactly leaves each client? «The data doesn't leave» is not enough: is it a raw gradient, an aggregate, something with added noise or with clipped updates? Who aggregates, and can they see contributions one by one? A server that sees every individual update is in a very different position from one that only sees an encrypted sum. Which threat model is being assumed? That is, who is allowed to be lying: a network observer, any client, the server itself. And what happens with dropouts and with who gets picked? If 5% or more falls off every round and only devices plugged in overnight take part, the model learns from a sample that is not the population.

That list depends on no particular technology and outlives the next wave of products. Federated learning is a serious architecture, with real deployments and a measurable contribution in communication; what it is not —and its own literature says so before anyone else— is a privacy guarantee that comes out of the box. The difference between the two readings is made by whoever asks what leaves, who aggregates, and what threat remains open.