Multi-Objective Optimization

Multi-objective optimization consists of simultaneously optimizing two or more objective functions that are in conflict, so that improving one usually means worsening another. That is why, in general, there is no single optimal solution but a set of trade-off solutions. The intuitive example is designing a car that is at once as fast and as cheap as possible: the two objectives pull in opposite directions.

Dominance and the Pareto front

The key conceptual tool is Pareto dominance. One solution dominates another if it is at least as good in all objectives and strictly better in at least one. A solution is Pareto-optimal (or non-dominated) when no other can improve one objective without degrading some other. The set of all such non-dominated solutions forms the Pareto front: the frontier of the best achievable trade-offs, where you can only gain in one objective at the cost of losing in another. As Deb and colleagues wrote, “in the absence of any further information, one of these Pareto-optimal solutions cannot be said to be better than the other.”

How it is tackled

There are several families of methods. Scalarization turns the problem into a single-objective one; its simplest version is the weighted sum, which assigns a weight to each objective and adds them—easy to implement, but unable to reach the non-convex regions of the front. The epsilon-constraint method optimizes one objective while treating the others as bounds, and it does capture those regions. And evolutionary algorithms work with a population of solutions, so they find many points of the front in a single run.

NSGA-II, a benchmark

The most cited evolutionary algorithm is NSGA-II, published by Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal and T. Meyarivan in 2002. It introduced a fast non-dominated sorting procedure, a crowding-distance mechanism to preserve the diversity of solutions and elitism, solving limitations of earlier algorithms. It is used in hyperparameter optimization, neural architecture search (balancing accuracy and cost) and design engineering.

The final decision is human

An important nuance: multi-objective optimization does not choose for you. It delivers the Pareto front, but selecting the specific solution to be implemented is a decision that falls to the person, according to their preferences and priorities. The technique illuminates the trade-offs; judgment resolves them.