ArXiv · 2026
Classical PAC-Bayesian generalization theory places prior distributions over Euclidean parameter space Θ ⊆ R^D, causing relative entropy to scale with network dimension D. For modern overparameterized neural networks where D ≫ n, this parameter-counting dependency yields vacuous generalization bounds (> 1,000%). In this work, we resolve this barrier by shifting prior specification from internal parameters to the non-parametric space of data distributions M₁(Z) via a Dirichlet Process (DP), DP(α, F₀), inducing parameter distributions through push-forward measures under the learning algorithm. By the Data Processing Inequality, parameter-space relative entropy is strictly bounded by the data-space DP divergence, which we evaluate in exact closed form using Digamma functions. The resulting PAC-Bayesian certificate is strictly invariant to network parameter count D, width, and depth. On intrinsic d^*-dimensional submanifolds, optimal partition scaling achieves the minimax non-parametric rate O(n^(-1/(d^*+2)) √(ln n)). Across vision architectures scaling up to 4.2 million parameters (D = 7,850 to D = 4,203,522) on the full MNIST benchmark, while classical weight-space bounds scale past 940% (becoming vacuous), our data-space DP-PAC-Bayes certificate remains stable, non-vacuous, and certified at textbf{8.8%--9.4%}.
Try inveni