Every multilayer perceptron is built on the same deal: the edges carry numbers and the nodes carry the nonlinearity. A weight multiplies a signal, a fixed activation such as ReLU or GELU bends it, and the network learns by adjusting the numbers. It works, but the deal is arbitrary. The Kolmogorov-Arnold representation theorem says there is another way to build a function, one where the edges themselves are learnable functions and the nodes only add. Kolmogorov-Arnold Networks, or KANs, take that theorem literally, and the result is a model that is more interpretable, more parameter-efficient on structured problems, and surprisingly good at rediscovering symbolic laws from data.
The theorem behind the architecture
The Kolmogorov-Arnold representation theorem, proved in 1957, states that any continuous multivariate function on a bounded domain can be written as a finite composition of continuous univariate functions and addition. In its two-layer form:
That is a striking claim. It says a function of n variables never needs a genuinely multivariate nonlinearity. It only needs univariate functions, applied to single inputs, combined by addition. The outer functions Φq and the inner functions ϕq,p depend on f, and the theorem says nothing about how smooth they are. In the worst case they can be fractal and pathological, which is why the theorem sat on the shelf for decades as a curiosity rather than a blueprint.
The KAN idea is to turn the theorem into a trainable architecture by restricting those univariate functions to a smooth, parameterised family: B-splines. If the true function is reasonably well behaved, a spline can approximate each univariate piece closely, and the network can learn the decomposition from data. The theorem supplies the structure; the splines supply the learnability.
What a KAN layer actually computes
An MLP layer is a linear map followed by a fixed nonlinearity applied at each node. A KAN layer inverts that arrangement. Every edge carries a learnable univariate function, and every node simply sums its incoming signals:
There is no activation function at the node at all. The nonlinearity lives entirely on the edges, inside the ϕl,j,i .
Each edge function is parameterised as a B-spline plus a smooth basis function:
Here b(x)=x/(1+e−x) is the SiLU basis, Bi are B-spline basis functions of order k on a grid of size G, and the coefficients ci are the learnable parameters. The grid controls resolution: a coarse grid gives a smooth, low-capacity function, while a fine grid can capture sharp features. One practical advantage is grid extension. A KAN trained on a coarse grid can be refined to a finer grid by fitting the finer spline to the learned function, without retraining from scratch. That is the analogue of increasing resolution in a numerical method, and it gives a principled way to add capacity during training.
Stacking layers composes these univariate functions, so a deep KAN can represent functions whose structure is compositional: a sum of functions of sums of functions, and so on. The width of a layer sets how many univariate pieces are available at that stage, just as width in an MLP sets how many features are available.
Why learnable edges help
The first payoff is interpretability. In an MLP, the learned object is a weight matrix, and understanding what a single weight means requires tracing it through the rest of the network. In a KAN, the learned object on each edge is a one-dimensional function, and a one-dimensional function can be plotted. You can look at an edge and see that it learned something close to a sine wave, a logarithm, or a saturating curve. After training, edges whose functions are near zero can be pruned, and the remaining functions can be fitted with symbolic expressions, turning the trained network into a readable formula. That is why KANs have attracted attention in scientific machine learning: when the data come from a physical law, the network has a chance of rediscovering the law rather than merely approximating it.
The second payoff is parameter efficiency on structured problems. A function with a compositional structure, such as f(x,y)=sin(xy) or a polynomial in a few variables, can require a large MLP to approximate to high accuracy, because the MLP must build the structure out of generic piecewise-linear pieces. A KAN can represent the same structure directly, with far fewer parameters. In the original KAN paper, small KANs matched or beat larger MLPs on symbolic regression tasks and on fitting special functions, often by orders of magnitude in parameter count.
The third payoff is a cleaner inductive bias for low-dimensional problems. Many scientific problems have few inputs and a smooth target with identifiable structure. An MLP treats such a problem as a generic high-dimensional approximation task. A KAN treats it as a composition of univariate pieces, which is a much stronger and often correct prior.
A small example: rediscovering a formula
The cleanest demonstration is symbolic regression. Suppose the data come from a function with multiplicative structure, and you want the model to recover it rather than just interpolate it. Using the pykan library, the workflow is short:
from kan import KAN import torch # synthetic data: f(x, y) = sin(x * y), a function with multiplicative structure torch.manual_seed(0) x = torch.rand(1000, 2) * 4 - 2 y = torch.sin(x[:, 0] * x[:, 1]).unsqueeze(1) model = KAN(width=[2, 5, 1], grid=5, k=3) model.fit({"train_input": x, "train_label": y}, steps=200, lamb=0.01) model.plot() # visualise every learned univariate function model.prune() # drop edges whose function is near zero model.symbolic_formula() # fit symbolic expressions to the learned splines Three things about this workflow are worth noting. First, model.plot() is not a debugging aid; it is the main interpretability tool, because each edge function is a plot. Second, pruning is meaningful in a way weight pruning in an MLP is not: removing an edge removes a whole univariate function, and the surviving graph is a readable decomposition of the target. Third, symbolic_formula() closes the loop from data to equation, which is the point of the exercise. If the learned functions are close to sine, exponential, or polynomial pieces, the symbolic fit recovers them, and you can check the recovered formula against held-out data.
The same pattern applies to real scientific data. Given measurements of a physical system, a KAN can be trained, pruned, and converted to a candidate formula, which a domain expert can then inspect, test, and refine. That is a different workflow from deploying a black-box predictor, and for science it is often the more valuable one.
Where KANs struggle
KANs are not a replacement for MLPs, and the honest limitations matter more than the headline results.
Training is slower. Evaluating a spline on every edge costs more than a matrix multiply followed by a pointwise activation, and the backward pass through the spline coefficients is heavier still. For large models, that cost dominates, and the parameter efficiency on small problems does not translate into a compute advantage at scale.
Scaling behaviour is unsettled. The original paper reported favourable neural scaling laws for KANs on some tasks, but follow-up work has shown MLPs catching up or overtaking KANs as model and data scale grow, particularly on high-dimensional tasks such as vision and language where the target function has little low-dimensional compositional structure to exploit. No KAN has yet matched a transformer on a large language or vision benchmark, and there is no evidence it should.
Optimisation is trickier. Spline coefficients interact with each other in ways that weights in an MLP do not, and the loss landscape can be harder to navigate. Grid extension helps, but choosing the grid size, spline order, and regularisation strength adds hyperparameters that an MLP does not have. Too fine a grid overfits; too coarse a grid underfits; and the right choice depends on the smoothness of the unknown target.
Finally, interpretability has a limit. A KAN is interpretable when the true function is compositional and the learned decomposition is sparse. On a genuinely high-dimensional problem with no such structure, the learned edge functions are arbitrary wiggles, and plotting them tells you nothing. Interpretability is a property of the problem as much as of the model.
When to reach for a KAN
Use a KAN when the problem is low-dimensional, the target is plausibly a composition of simple univariate functions, and you care about the formula as much as the prediction. Scientific modelling, symbolic regression, and small tabular problems with smooth structure are the natural home. Avoid KANs when the inputs are high-dimensional raw signals, when training speed at scale matters, or when a transformer or a large MLP is already known to work well. In those settings, the MLP's simplicity and speed win.
A practical checklist before training: start with a small width and a coarse grid, and refine the grid only if the fit is underfitting; use the L1-style regularisation on edge magnitudes that the library provides, so pruning has something to work with; plot the edge functions early, not just at the end, because a healthy KAN shows smooth, recognisable curves while a struggling one shows noise; and always validate the symbolic formula on held-out data before believing it, since a spline can fit training noise with a plausible-looking curve.
Takeaway
MLPs put the learning in the weights and the nonlinearity in fixed activations. KANs put the learning in the functions themselves, with one learnable univariate function per edge and nothing but addition at the nodes. The Kolmogorov-Arnold theorem guarantees that such a decomposition exists; B-splines make it trainable. The result is not a universal replacement for the MLP, but a sharper tool for a specific job: low-dimensional, structured problems where the learned function should be inspected, pruned, and ideally read back as a formula. When the problem fits that description, a KAN can do with a handful of parameters what an MLP needs thousands for, and it can show its working.
Connect with me on LinkedIn: linkedin.com/in/umeshtharukaofficial

