Necessary and Sufficient Conditions for Universal Approximation in KAN Networks Have Been Proven

Introduction: A Major Breakthrough in the Theoretical Foundation of KAN Networks

Since Kolmogorov-Arnold Networks (KAN) burst onto the scene in 2024, this novel neural network architecture grounded in classical mathematical theorems has attracted widespread attention across the AI research community. Unlike traditional Multi-Layer Perceptrons (MLPs) that apply activation functions at nodes, KAN places learnable functions on the network's "edges," demonstrating remarkable expressive power and interpretability.

However, a fundamental theoretical question had remained unresolved: What conditions must edge functions satisfy for KAN networks to achieve Universal Approximation? Recently, a paper published on arXiv (arXiv:2604.23765v1) provided a precise and elegant answer, laying a solid cornerstone for KAN's theoretical edifice.

Core Finding: One Non-Affine Function Is All You Need

The paper's central question can be summarized in one sentence: if all edge functions in a KAN network are affine functions (i.e., linear transformations of the form f(x) = ax + b), the network obviously cannot approximate arbitrary continuous functions — it would merely be a stack of linear transformations with extremely limited expressive power. So, beyond affine functions, how many non-affine functions are needed to guarantee universal approximation?

The researchers' answer is stunning: Just one.

More precisely, the paper proves the following key theorem: for deep KAN networks, if all edge functions are either affine functions or equal to some fixed continuous function σ, then the network possesses universal approximation capability if and only if σ is a non-affine function. In other words, as long as even one non-affine continuous function exists as an edge function, a deep KAN can approximate any continuous function to arbitrary precision.

The elegance of this conclusion lies in its simultaneous provision of both "sufficient" and "necessary" conditions:

• Sufficiency: One non-affine function plus affine functions is enough to achieve universal approximation
• Necessity: If all edge functions are affine, universal approximation inevitably fails

Technical Analysis: Why Is This Conclusion So Important?

A Profound Echo of the MLP Universal Approximation Theorem

Readers familiar with neural network theory will immediately recall the classical MLP Universal Approximation Theorem. In 1989, Cybenko, Hornik, and others proved that MLPs with a single hidden layer can approximate any continuous function when the activation function meets certain conditions. Subsequently, researchers further demonstrated that for deep networks, piecewise linear activation functions such as ReLU also possess universal approximation capability.

This new KAN result forms a beautiful symmetry with MLP theory. In MLPs, the key lies in the nonlinearity of the activation function; in KANs, the key similarly lies in the "non-affinity" of edge functions. The two architectures arrive at the same destination via different paths, both revealing a profound truth: The expressive power of neural networks stems from the introduction of nonlinearity, and the 'amount' of nonlinearity required can be vanishingly small — a single nonlinear element suffices.

Practical Guidance for KAN Design

This theoretical result carries significant practical implications for KAN design:

1. Simplified architecture search: Designers need not equip every edge with complex nonlinear functions. In theory, using simple affine transformations for most edges and introducing non-affine functions only at key positions is sufficient to guarantee the network's expressive power.

2. Computational efficiency optimization: The computational overhead of affine functions is far lower than that of spline functions or other complex parameterized functions. This finding suggests a "sparse nonlinearity" KAN design strategy that could dramatically reduce computational costs while maintaining expressive power.

3. Theoretical guarantee: B-splines, Chebyshev polynomials, wavelet functions, and other commonly used KAN edge functions are all obviously non-affine. This theorem provides rigorous theoretical endorsement for these choices.

The Critical Role of Depth

It is particularly worth noting that the paper's conclusions pertain to "deep" KAN networks. This means network depth plays an indispensable role in universal approximation. A single non-affine function, through repeated composition and stacking in deep networks, can progressively build increasingly complex function representations layer by layer. This is highly consistent with the empirical understanding in deep learning that "depth is preferable to width," and provides theoretical proof for the value of depth in KAN networks.

Academic Background and Significance

The name Kolmogorov-Arnold Networks originates from the Superposition Theorem (Kolmogorov-Arnold Representation Theorem) proven by Soviet mathematicians Kolmogorov and Arnold in 1957. The theorem states that any multivariate continuous function can be represented as a superposition of finitely many univariate continuous functions. KAN is precisely a modern interpretation of this classical theorem within the deep learning framework.

However, while the original Kolmogorov-Arnold theorem guarantees the existence of such representations, the univariate functions involved may be extremely irregular and difficult to learn in practice. Modern KANs circumvent this issue through parameterized edge functions (such as B-splines) and deep stacking. The contribution of this paper is that it precisely characterizes the "minimum requirements" these edge functions must satisfy, thus taking a critical step toward theoretical completeness.

This research also represents a "cross-disciplinary" achievement, spanning approximation theory, functional analysis, and deep learning theory, showcasing the rich fruits of interdisciplinary research between mathematics and AI.

Outlook: KAN Theory Approaching Maturity

This paper marks KAN theoretical research entering a new phase. From the initial question of "can it work" to the current "why does it work" and "what is the minimum requirement," KAN's theoretical foundation is rapidly maturing.

Future directions worth watching include:

• Approximation rate analysis: Universal approximation only answers the qualitative question of "can it approximate." The next step is to answer the quantitative question of "how fast does it approximate." Do different types of non-affine functions yield different approximation efficiencies?
• Finite width versus finite depth trade-offs: In practical finite networks, how do width and depth affect approximation accuracy?
• Integration with regularization: Can the theoretical discovery of sparse nonlinearity inspire new regularization methods that improve generalization in practical training?

From a broader perspective, as an alternative architecture to MLPs, the refinement of KAN's theoretical framework will provide important intellectual reserves for the "post-MLP era" of deep learning. When theory and practice form a virtuous cycle, KAN has the potential to unleash even greater capabilities in scientific computing, interpretable AI, and other domains.