Learning & Remembering

This paper separates learning into two ingredients. A learning rule decides how an agent would like to change its decisions in response to feedback; a memory rule realizes that change by adjusting the agent's parameters. The distinction sounds tautological, but it isn't: in almost any parameterized system, the memory rule cannot perfectly implement the learning rule's request. We argue that the fidelity of the memory rule to the learning signal is the natural quantity to study when analyzing what a learning system can and cannot achieve.

To make this precise, we introduce differential expressivity: the set of decision rules an agent can reach with a small update to its parameters. Geometrically, it is the tangent space of the architecture in function space — the linear span of the partial derivatives of the parameterization with respect to each parameter, evaluated at the current parameters. Whereas classical expressivity results describe every function an architecture could represent in principle, differential expressivity describes only the updates it can achieve locally. It is a strictly smaller and, we argue, more relevant quantity for analyzing a learning agent in motion.

Given an inner product on function space, we can measure both the size of an update and its cosine alignment with the learning signal. Our main result shows that, to first order, the change in performance under an update $h$ factors as

An agent controls two of these three factors: the alignment $\cos(\nabla_f P, h)$ and the magnitude $\|h\|$. The functional gradient $\nabla_f P$ is a property of the problem. When step size is bounded, the only thing left to improve is alignment — and alignment itself is bounded above by the alignment of the differential expressivity with the gradient. That upper bound is a property of the architecture, invisible to memory-rule techniques such as natural gradients.

This decomposition disentangles a few things that are usually conflated. A vanishing gradient in parameter space could mean the functional gradient is zero (a local optimum of the problem), the update magnitude is zero (a null direction of the architecture), or the alignment is zero (a local optimum of the architecture). It also suggests an alignment-based definition of plasticity: rather than treating plasticity as a global, objective-agnostic property of an architecture, we define the directed plasticity of an agent in a direction $c$ as $\cos(c, h)$, the alignment of the update induced by $c$ with $c$ itself. This gives a plasticity quantity that depends on what the agent is being asked to learn.

Learning & Remembering: Differential Expressivity Constrains Learning, joint work with Bruno Castro da Silva, was accepted at the Finding the Frame workshop at RLC 2026.

• Finding the Frame (RLC 2026)