首页 时政热点 科技头条 智能AI 安全攻防 数码硬件 开发者生态 归档 标签 关于
智能AI morning

深度单项式网络中的奇异学习和奥卡姆剃刀

2026-06-30 1 阅读 约2分钟阅读 Kathl\'en Kohn, Giovanni Luca Marchetti, Farhan Shabir, Vahid Shahverdi, Weisheng Wang
分享:
字号:
arXiv:2606.28464v1 Announce Type: new Abstract: In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture. These critical points occur where the Jacobian of the model's parametrization is rank-deficient, and are the most pronounced singularities studied in Singular Learning Theory.我们通过多项式代数工具(例如梅森定理)研究具有单项式激活的深度全连接网络中的此类点。 We show that, for sufficiently large activation degree, criticality occurs precisely at subnetworks, i.e., at parameter configurations where some neurons are inactive or redundant. This offers a mathematical perspective on the implicit bias in deep neural networks, explaining the tendency of these models to converge toward simpler functions.
这篇文章对您有帮助吗?

订阅66必读

每日精选科技资讯,直达你的邮箱