論文 / arXiv:2610.31663
Phase Transition from Random Features to Fully Trainable Networks
PLAIN SUMMARY / やさしい要約
ニューラルネットには、最初から決め打ちの部品(固定)を多く使うやり方と、全部を学習させるやり方があります。この研究は、その中間として「一部だけ学習できる」作りにすると、最初はよく改善するのに、途中から伸びが小さくなる傾向が理論と実験の両方で見えることを示します。どれだけ学習できる部品を増やすと効きやすいかを、Pという割合と、初期状態のずれ具合を表す量で説明します。たとえるなら、調整できるツマミを少しだけ増やすと最初は音が整うが、ツマミを増やし続けてもだんだん効果が小さくなる、という現象に近い読み取りができます。
AIが専門用語を使わずに書いた解説です。内容の正確さは、下の原文の要旨で確認してください。
ABSTRACT / 要旨(原文)
Constructing accurate and computationally tractable approximation spaces is a fundamental task in scientific computing. Partially trainable neural networks bridge fixed random features and fully adaptive representations. This paper quantifies a phase transition pattern in approximation bounds and observes analogous behavior in training errors: a rapid but smooth initial improvement followed by diminishing returns, with a sharper transition under greater initialization mismatch. Under polynomial spectral decay with exponent $α>1$ and leverage compatibility, we establish a squared $L_2$ approximation rate $\mathcal{O}((κ_P\log(m/δ)/m)^α)$ for two-layer networks of width $m$ and trainable proportion $P$, where $κ_P=(P/κ_1^*+(1-P)/κ_0^*)^{-1}$. The bound holds uniformly over the unit ball of a reproducing kernel Hilbert space (RKHS) with probability at least $1-δ$. The constants $κ_0^*$ and $κ_1^*$ quantify the fixed and trainable groups' sampling mismatches relative to the target RKHS. For spherical biased-ReLU features in dimension $d$, we verify these assumptions with $α=(d+2)/(d-1)$ and $κ_0^*=4\sqrt{2}τ(d+1)/d$ for initialization scale $τ\ge1$, and $κ_1^*=1$ under ridge leverage sampling. For $d=5$ and $τ=10$, a trainable proportion of approximately $4.1\%$ captures $90\%$ of the reduction in the approximation bound from $P=0$ to $P=1$. A structured three-layer example illustrates how a few trainable first-layer neurons exploit fixed downstream blocks through nonlinear composition. Numerical experiments across diverse settings, including networks with up to 15 layers, exhibit similar phase transitions in training errors.
ここに表示しているのは原論文の要旨です。AIによる要約や解釈は含みません。
