Tyler Farghly

selected papers

1. 

   Metropolis-Adjusted Diffusion Models

   K Lam^*, T Farghly^*, C Williams^*, J Yang, Y Teh, and A Doucet

   In NeurIPS 2026

   Abs URL Bib

   Sampling from score-based diffusion models incurs bias due to both time discretisation and the approximation of the score function. A common strategy for reducing this bias is to apply corrector steps based on the unadjusted Langevin algorithm (ULA) at each noise level within a predictor-corrector framework. However, ULA is itself a biased sampler, as it discretises a continuous diffusion process. In this work, we consider adjusted Langevin correctors that employ Metropolis–Hastings (MH) or Barker’s accept-reject steps to correct for this bias. Since the target density ratio typically required by MH-based algorithms is unavailable, we propose methods that instead utilise the score function to compute the correct acceptance probability. We introduce the first exact method for adjusting Langevin corrections in diffusion models, based on a two-coin Bernoulli factory algorithm. We also propose an efficient approximation based on Simpson’s rule that achieves accuracy of order 5/2 in the step size at near-zero marginal cost. We demonstrate that these procedures improve sample quality on both synthetic and image datasets, yielding consistent gains in Fréchet Inception Distance (FID) on the latter.

   @inproceedings{farghly2025diffusion,
     title = {Metropolis-Adjusted Diffusion Models},
     author = {Lam, Kevin H and Farghly, Tyler and Williams, Christopher and Yang, Jun and Teh, Yee Whye and Doucet, Arnaud},
     booktitle = {The Fortieth Annual Conference on Neural Information Processing Systems},
     year = {2026},
   }

2. 

   Benign Overfitting Does Not Occur in Diffusion Models

   T Farghly^*, B Dupuis^*, A Durmus, and U Simsekli

   arXiv 2026

   Abs URL Bib

   Benign overfitting and double descent have come to shape our understanding of generalization in deep learning, establishing that overfitting is not only compatible with good generalization but can actively benefit it. Diffusion models share much of the machinery of standard deep learning, so it is natural to assume that they also exhibit these properties. In this work, we show that this assumption is largely incorrect. We first establish fundamental impossibility results showing that, unless the sample size grows exponentially with the data dimension, overfitting and good generalization cannot occur simultaneously. Consequently, the population loss follows a classical U-shaped curve in model complexity rather than exhibiting double descent. Analyzing a simplified setting, we identify a key difference between regression and score matching: regression benefits from an alignment between the target and the empirical covariance; score matching admits no such alignment, leaving overfitting irreparably harmful. We further identify implicit regularization stemming from time-smoothness of the score and early stopping during training as mechanisms that prevent such overfitting and verify our findings with high-dimensional image generation experiments. Our results reveal that generalization in diffusion models is governed by mechanisms distinct from those of traditional regression, motivating the development of new theory.

   @article{farghly2026benign,
     title = {Benign Overfitting Does Not Occur in Diffusion Models},
     author = {Farghly, Tyler and Dupuis, Benjamin and Durmus, Alain and Simsekli, Umut},
     journal = {arXiv preprint arXiv:2607.02671},
     year = {2026},
   }

3. 

   Beyond Real Data: Synthetic Data through the Lens of Regularization

   A Shidani, T Farghly, Y Sun, H Ganjgahi, and G Deligiannidis

   In AISTATS 2026 (Oral)

   Abs URL Bib

   Synthetic data can improve generalization when real data is scarce, but excessive reliance may introduce distributional mismatches that degrade performance. In this paper, we present a learning-theoretic framework to quantify the trade-off between synthetic and real data. Our approach leverages algorithmic stability to derive generalization error bounds, characterizing the optimal synthetic-to-real data ratio that minimizes expected test error as a function of the Wasserstein distance between the real and synthetic distributions. We motivate our framework in the setting of kernel ridge regression with mixed data, offering a detailed analysis that may be of independent interest. Our theory predicts the existence of an optimal ratio, leading to a U-shaped behavior of test error with respect to the proportion of synthetic data. Empirically, we validate this prediction on CIFAR-10 and a clinical brain MRI dataset. Our theory extends to the important scenario of domain adaptation, showing that carefully blending synthetic target data with limited source data can mitigate domain shift and enhance generalization. We conclude with practical guidance for applying our results to both in-domain and out-of-domain scenarios.

   @inproceedings{shidani2026beyond,
     title = {Beyond Real Data: Synthetic Data through the Lens of Regularization},
     author = {Shidani, Amitis and Farghly, Tyler and Sun, Yang and Ganjgahi, Habib and Deligiannidis, George},
     booktitle = {The 29th International Conference on Artificial Intelligence and Statistics},
     year = {2026},
     url = {https://openreview.net/forum?id=Y6Jm6KMl6B},
     special = {Oral},
   }

4. 

   Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive

   T Farghly^*, P Potaptchik^*, S Howard^*, G Deligiannidis, and J Pidstrigach

   In NeurIPS 2025

   Abs URL Bib

   Diffusion models have achieved state-of-the-art performance, demonstrating remarkable generalisation capabilities across diverse domains. However, the mechanisms underpinning these strong capabilities remain only partially understood. A leading conjecture, based on the manifold hypothesis, attributes this success to their ability to adapt to low-dimensional geometric structure within the data. This work provides evidence for this conjecture, focusing on how such phenomena could result from the formulation of the learning problem through score matching. We inspect the role of implicit regularisation by investigating the effect of smoothing minimisers of the empirical score matching objective. Our theoretical and empirical results confirm that smoothing the score function—or equivalently, smoothing in the log-density domain—produces smoothing tangential to the data manifold. In addition, we show that the manifold along which the diffusion model generalises can be controlled by choosing an appropriate smoothing.

   @inproceedings{farghly2025diffusioo,
     title = {Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive},
     author = {Farghly, Tyler and Potaptchik, Peter and Howard, Samuel and Deligiannidis, George and Pidstrigach, Jakiw},
     booktitle = {The Thirty-ninth Annual Conference on Neural Information Processing Systems},
     year = {2025},
     url = {https://openreview.net/forum?id=4JihzQXNJn},
   }

5. 

   Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis

   T Farghly, P Rebeschini, G Deligiannidis, and A Doucet

   In ICLR 2026

   Abs URL Bib

   The success of denoising diffusion models raises important questions regarding their generalisation behaviour, particularly in high-dimensional settings. Notably, it has been shown that when training and sampling are performed perfectly, these models memorise training data—implying that some form of regularisation is essential for generalisation. Existing theoretical analyses primarily rely on algorithm-independent techniques such as uniform convergence, heavily utilising model structure to obtain generalisation bounds. In this work, we instead leverage the algorithmic aspects that promote generalisation in diffusion models, developing a general theory of algorithm-dependent generalisation for this setting. Borrowing from the framework of algorithmic stability, we introduce the notion of score stability, which quantifies the sensitivity of score-matching algorithms to dataset perturbations. We derive generalisation bounds in terms of score stability, and apply our framework to several fundamental learning settings, identifying sources of regularisation. In particular, we consider denoising score matching with early stopping (denoising regularisation), sampler-wide coarse discretisation (sampler regularisation), and optimising with SGD (optimisation regularisation). By grounding our analysis in algorithmic properties rather than model structure, we identify multiple sources of implicit regularisation unique to diffusion models that have so far been overlooked in the literature.

   @inproceedings{farghly2026implicit,
     title = {Implicit Regularisation in Diffusion Models: An Algorithm-Dependent Generalisation Analysis},
     author = {Farghly, Tyler and Rebeschini, Patrick and Deligiannidis, George and Doucet, Arnaud},
     booktitle = {The Fourteenth International Conference on Learning Representations},
     year = {2026},
     url = {https://openreview.net/forum?id=3lSqgESdPu},
   }

6. 

   Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincaré Inequality

   A Mousavi-Hosseini^*, T Farghly^*, Y He, K Balasubramanian, and M Erdogdu

   In COLT 2023

   Abs URL Bib

   Langevin diffusions are rapidly convergent under appropriate functional inequality assumptions. Hence, it is natural to expect that with additional smoothness conditions to handle the discretization errors, their discretizations like the Langevin Monte Carlo (LMC) converge in a similar fashion. This research program was initiated by Vempala and Wibisono (2019), who established results under log-Sobolev inequalities. Chewi et al. (2022a) extended the results to handle the case of Poincaré inequalities. In this paper, we go beyond Poincaré inequalities, and push this research program to its limit. We do so by establishing upper and lower bounds for Langevin diffusions and LMC under weak Poincaré inequalities that are satisfied by a large class of densities including polynomially-decaying heavy-tailed densities (i.e., Cauchy-type). Our results explicitly quantify the effect of the initializer on the performance of the LMC algorithm. In particular, we show that as the tail goes from sub-Gaussian, to sub-exponential, and finally to Cauchy-like, the dependency on the initial error goes from being logarithmic, to polynomial, and then finally to being exponential. This three-step phase transition is in particular unavoidable as demonstrated by our lower bounds, clearly defining the boundaries of LMC.

   @inproceedings{pmlr-v195-mousavi-hosseini23a,
     title = {Towards a Complete Analysis of Langevin Monte Carlo: Beyond Poincaré Inequality},
     author = {Mousavi-Hosseini, Alireza and Farghly, Tyler K. and He, Ye and Balasubramanian, Krishna and Erdogdu, Murat A.},
     booktitle = {Proceedings of Thirty Sixth Conference on Learning Theory},
     pages = {1--35},
     year = {2023},
     editor = {Neu, Gergely and Rosasco, Lorenzo},
     volume = {195},
     series = {Proceedings of Machine Learning Research},
     month = {12--15 Jul},
     publisher = {PMLR},
   }

7. 

   Time-independent Generalization Bounds for SGLD in Non-convex Settings

   T Farghly, and P Rebeschini

   In NeurIPS 2021

   Abs URL Bib

   We establish generalization error bounds for stochastic gradient Langevin dynamics (SGLD) with constant learning rate under the assumptions of dissipativity and smoothness, a setting that has received increased attention in the sampling/optimization literature. Unlike existing bounds for SGLD in non-convex settings, ours are time-independent and decay to zero as the sample size increases. Using the framework of uniform stability, we establish time-independent bounds by exploiting the Wasserstein contraction property of the Langevin diffusion, which also allows us to circumvent the need to bound gradients using Lipschitz-like assumptions. Our analysis also supports variants of SGLD that use different discretization methods, incorporate Euclidean projections, or use non-isotropic noise.

   @inproceedings{NEURIPS2021_a4ee59dd,
     author = {Farghly, Tyler and Rebeschini, Patrick},
     booktitle = {Advances in Neural Information Processing Systems},
     editor = {Ranzato, M. and Beygelzimer, A. and Dauphin, Y. and Liang, P.S. and Vaughan, J. Wortman},
     pages = {19836--19846},
     publisher = {Curran Associates, Inc.},
     title = {Time-independent Generalization Bounds for SGLD in Non-convex Settings},
     volume = {34},
     year = {2021},
   }