Particle Swarm Optimization

Particle Swarm Optimization (PSO) is an optimization metaheuristic proposed by James Kennedy and Russell Eberhart in 1995. It draws on the collective behavior of bird flocks and fish schools: a set of candidate solutions, called particles, moves through the search space and adjusts its trajectory according to its own experience and that of the group. PSO does not guarantee that the global optimum will be found; through successive iterations it seeks a solution that is good enough.

Each particle represents a possible solution to the problem and is defined by two vectors: its position in the search space and its velocity, which sets the direction and magnitude of its next move.

Velocity and position update

On each iteration, PSO first updates every particle's velocity and then its position. The new velocity combines three terms: inertia, which retains part of the previous velocity; a cognitive term that pulls the particle toward the best position it has personally found so far (pbest); and a social term that pulls it toward the best position found by the whole swarm (gbest). Two random numbers weight the cognitive and social terms at each step, adding variability to the search. Once the velocity is computed, the position is updated by adding that vector, so the particle moves to its new point. The balance among inertia, personal attraction, and collective attraction governs the trade-off between exploring new regions and exploiting the best ones already known.

Algorithm parameters

PSO's behavior depends on a handful of parameters. The inertia weight w controls how much previous velocity is retained: high values favor global exploration, low values a finer local search. The cognitive coefficient c1 weights the pull toward the personal best, and the social coefficient c2 the pull toward the global best; the two are usually set equal. The swarm size —the number of particles— determines how much of the space is covered in parallel, at the cost of more computation. In 1998, Yuhui Shi and Russell Eberhart published a modified PSO and a parameter-selection study. The latter's public abstract confirms that they analyzed the effects of inertia weight and maximum velocity and offered guidelines; the full text is behind a subscription, so that source does not let us verify the w≈0.7 and c1=c2≈1.5 recipe here. The parameters should be treated as problem-dependent, not universal constants.

Convergence limits and uses

PSO offers no guarantee of optimality. Its main weakness is premature convergence: if the swarm clusters too soon around a local optimum, it loses diversity and stagnates without reaching the global optimum. Performance is also sensitive to the parameters —a poorly chosen inertia weight or coefficients degrade the search— and no universal configuration exists. Later variants, such as a decreasing inertia weight or the constriction factor, ease these problems but do not remove them. With those caveats, PSO is applied to tuning neural networks, engineering design, scheduling, and continuous optimization problems where a gradient is unavailable. One frequent confusion is worth flagging: PSO is an optimization metaheuristic and has no connection to boosting or to ensemble methods in machine learning.