Our paper was awarded oral in AISTAS 2026! Beyond Real Data: Synthetic Data through the Lens of Regularization