Preprocessing Without Leaking the Future: Fit and Transform

As of July 30, 2026, the most dangerous preprocessing error is not choosing the mean instead of the median. It is allowing validation data, the future, or the target to influence a transformation before evaluation. The model then receives clues unavailable at use time and appears better than it is. The durable rule fits on one line: split first; fit every transformation only on training data; then transform validation, test, and production without learning again.

Begin with the unit and prediction time

Before cleaning, define what one row represents, when the prediction is made, and which information existed at that moment. If the target is loan default at approval time, a variable recorded after the due date belongs to the future. If several rows belong to one person, random splitting may place that person in both train and test.

The split should match the process: by time when predicting the future, by patient or user for repeated measurements, and by site for geographic generalization. Only then should training data be inspected for decisions. Test data remains sealed until the pipeline is fixed.

Fit learns; transform applies

A standardizer learns a mean and standard deviation; an imputer learns medians or a model; an encoder learns categories; PCA learns axes. Fit estimates those parameters. Transform applies the parameters already fixed. Calculating them over the complete dataset leaks validation information even when labels were never explicitly used.

Scikit-learn’s guide to common pitfalls documents inconsistent preprocessing and data leakage. Its Pipeline object chains transformations and estimator so cross-validation fits each step inside each training fold. The tool helps, but temporal or grouped splitting remains an analyst’s decision.

Scaling: which statistic must survive

Standardization subtracts the training mean and divides by its standard deviation. Min-max scaling uses training minima and maxima to construct a range. Units stop dominating distance- or gradient-sensitive algorithms, while decision trees often need scaling less. A future value may fall outside 0–1; clipping or accepting it is a policy rather than a mathematical error.

Outliers influence means and extremes. A robust scaler based on medians and quantiles may be steadier, but first decide whether the value is an error, a legitimate rare event, or a regime change. Production should log how many values fall outside learned ranges because that is evidence of drift.

Categories: vocabulary and unknowns

One-hot encoding creates one column per category learned in training. Production needs a policy for unseen categories: error, unknown bucket, hashing, or another representation. Identifier order must never create a false distance. A categorical embedding learns vectors, but needs enough examples and does not remove handling for novel values.

Target encoding is especially delicate: it replaces a category with label statistics. Calculating it from the same row or whole set leaks the outcome. CatBoost proposed ordered target statistics to reduce this bias. In a general pipeline, category means should be calculated out of fold and always within training data.

Missingness: the reason may carry information

A value may be missing because of random equipment failure, because of an observed variable, or because of the unobserved value itself. These situations are commonly summarized as MCAR, MAR, and MNAR and imply different assumptions. Mean imputation does not magically recover missing information; it reduces variation and can distort relationships. A missingness indicator lets the model use the pattern, but may also encode an artifact of collection.

The MICE implementation chains conditional models and creates multiple imputations to represent uncertainty under explicit assumptions. Prediction may use a simpler technique, but it should be fitted only on training data, compared with a baseline, and evaluated by subgroup. If a variable is nearly always missing, the honest remedy may be to stop pretending it is observed.

Anomaly does not mean rubbish

An extreme point may be a sensor error, fraud, a rare patient, or exactly the event of interest. Deleting it because it is “far away” removes signal. Start with domain validity rules: a negative age may be corrected or excluded; a huge purchase requires investigation.

DBSCAN groups dense regions and marks points outside them under a chosen scale and parameters. It helps explore structure, but is not a universal quality judge. Distances behave differently in high dimensions and scaling changes neighborhoods. Any anomaly rule is fitted on training data and tested against known cases.

PCA compresses for use; t-SNE projects for inspection

PCA finds orthogonal directions of maximum variance and can transform new examples with learned axes. It may reduce collinearity or cost, but maximum variance is not maximum target signal. Component count belongs inside validation, with prior scaling preserved.

t-SNE was designed to visualize high-dimensional data by preserving probabilistic local neighborhoods in two or three dimensions. Global distances, cluster sizes, and empty spaces should not be read as faithful measures. Results change with perplexity, initialization, and seed, and the basic method does not provide a stable production transform. A t-SNE plot is an exploratory view, not a general compression stage.

Feature selection can leak too

Selecting columns through target correlation, Random Forest importance, or model performance uses labels. When performed before cross-validation, every fold already knows which features looked good elsewhere. Selection belongs inside the pipeline and repeats within each training fold.

The same applies to automated search, indirectly supervised autoencoders, or feature generation. Candidate history should be preserved to avoid trying until chance looks favorable. A feature is judged by out-of-sample gain, stability, computation cost, and availability at the real decision time.

Training and production must run the same transform

Copying logic between a notebook and service creates training-serving skew: names, time zones, defaults, or versions diverge. Research on data validation for production ML shows how schemas and statistics help detect anomalies and differences between environments.

Code, fitted parameters, column order, and dependencies are versioned with the model. The pipeline accepts raw data under the same contract as production. Tests with known records verify exact output; monitoring compares ranges, new categories, missingness rates, and distributions.

Preprocessing does not “remove” bias

Balancing classes or reweighting groups changes a distribution, but fairness depends on population, target, labels, and error cost. Cleaning may worsen disparities if it deletes underrepresented-group data as “anomalous.” Decisions and disaggregated metrics should be reported before and after.

Datasheets for Datasets proposes documenting motivation, composition, collection, preprocessing, uses, and maintenance. Provenance makes transformations auditable. “Clean data” is not a neutral state; it means someone applied rules with consequences.

A minimum reproducible pipeline

Define the target and cutoff; split by time, entity, or group; seal the test; in each fold fit imputation, categories, scaling, reduction, and selection on training data; apply to validation; choose; refit the complete pipeline on training plus validation if the protocol permits; evaluate once on test; serialize transform and model; monitor the input contract.

For an audit, choose one statistic—such as the mean used for scaling—and ask which rows produced it. If a future observation, test row, or post-target information appears, there is leakage. The technique can change; temporal discipline does not: first separate what the system could know, then allow each transformation to learn.

A second diagnostic is to permute the target after splitting and rerun the entire pipeline. Performance should collapse toward chance. If it remains implausibly strong, a feature, transform, duplicate, or grouping rule is carrying target information across the boundary. This negative control is cheap, repeatable, and often more revealing than another model comparison.