The first part of the thesis is devoted to McKean–Vlasov particle systems, which describe interacting particles whose dynamics depend on their collective distribution. These systems have recently become central in machine learning applications, including optimization and sampling algorithms. They are typically analyzed in the mean-field limit, where the number of particles tends to infinity and the system can be approximated by a McKean–Vlasov equation. However, the thesis focuses on a more realistic and challenging problem: understanding the long-time behavior of McKean–Vlasov particle systems with a fixed number of particles. We show that, under mild assumptions, the empirical distribution of particles converges toward the set of stationary solutions of the associated McKean–Vlasov equation. This is achieved without relying on strong assumptions, such as those used in current approaches, which are often difficult to verify in practice. We then specialize our analysis to two important algorithms: Consensus-Based Optimization (CBO) and Stein Variational Gradient Descent (SVGD). For CBO, a gradient-free optimization method, we establish long-time convergence with explicit rates, showing that particles concentrate near the global minimizer after many iterations. In the case of SVGD, we propose a noisy modification of the algorithm, which ensures convergence toward the target distribution.
The second part of the thesis focuses on Importance Sampling particle systems, in which particles are generated sequentially and assigned importance weights to approximate a target distribution. A major difficulty in this framework is weight degeneracy, which occurs when most weights are negligible while only a few dominate. As a result, only a small number of particles have a significant influence on the approximation of the target measure. To address this issue, we introduce a modified kernel-based adaptive Importance Sampling algorithm that applies a transformation to the weights, reducing their variance while preserving convergence toward the target distribution. This approach improves performance, particularly in high-dimensional settings. In addition, we develop a novel method for gradient-free optimization. While many current approaches consist of selecting the best sample from a given algorithm, the proposed method instead uses a weighted average based on Importance Sampling. We demonstrate the efficiency of our approach by comparing it to standard random search and show that, under suitable assumptions, it achieves improved convergence rates while maintaining the same computational complexity.
Overall, the thesis provides new theoretical insights into the long-time behavior of stochastic particle systems and demonstrates their effectiveness in a wide range of applications. It bridges the gap between theory and practice by offering improved algorithms for sampling and optimization, supported by rigorous convergence guarantees.