arXiv：cs.LG· Hanli Xu, Fengxiang He, Sarat Moka·· 5 小时前AI 评分33
超越全局散度：贝叶斯推理的局部质量视角
Beyond Global Divergences: A Local-Mass Perspective on Bayesian Inference
AI 导读
该论文提出 Mass Index 与正则化扩展 KL（RE-KL）两种数学工具，用于刻画 KL 散度、ELBO 等全局目标无法直接捕捉的分布"局部质量行为"。研究证明局部 RE-KL 在两种 KL 方向下满足绝对、相对与方向性不等式，并给出贝叶斯更新改变局部质量的理论解释。实验显示该方向性差异在贝叶斯神经网络变分后验中依然可见，最大规模达 ImageNet 上的 ResNet-50，代码已开源。
正文
Abstract:Global objectives, such as KL divergence and ELBO, are widely used in Bayesian inference for measuring distributional discrepancy. This paper studies distributional ``local-mass behaviours'' that are not directly captured by such global objectives. We introduce and use two mathematical tools: (1) Mass Index for recording the polynomial and logarithmic decay scales of local mass, and (2) regularised extended KL (RE-KL), a set-localised divergence that can be formulated in the presence of singular components. Mass Indices help characterise how Bayesian updating changes local mass: (1) power-log likelihood factors shift it explicitly, and (2) parameter-dependent supports, or their smooth softenings, may change the local scale through the amount of mass that remains near the parameter value. Using local RE-KL, we prove absolute, relative, and directional inequalities for comparing local small-ball masses under the two KL directions. Together, these results provide a local theoretical account of local mass behaviour. Experiments provide controlled illustrations of the local behaviour, and show that the directional comparison remains visible in the variational posteriors of Bayesian neural networks up to ResNet-50 on ImageNet. Code is available at this https URL.
| Comments: | 32 pages, 9 figures, 4 tables, including appendices |
| Subjects: | Machine Learning (stat.ML); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.27090 [stat.ML] |
| (or arXiv:2606.27090v2 [stat.ML] for this version) | |
| https://doi.org/10.48550/arXiv.2606.27090 arXiv-issued DOI via DataCite |
Submission history
From: Hanli Xu [view email]
[v1]
Thu, 25 Jun 2026 14:26:42 UTC (140 KB)
[v2]
Fri, 2 Oct 2026 02:03:45 UTC (377 KB)
来源：arXiv：cs.LG · arxiv.org