I trained a 548M-parameter masked diffusion language model in which the three places a discrete model touches its tokens are all replaced by wave mechanics. Tokens are unit-magnitude wave packets. Forward diffusion is phase decoherence. The sequence mixer is an attention-free split-step integrator of a Schrödinger equation. Everything else (the MDLM objective, the sampler, the evaluation protocol) is standard, so anything that differs from a transformer baseline is attributable to the substrate rather than to the training setup.

The weights are on Hugging Face, with an interactive demo that visualises the wavefield as it denoises.

It writes fluent, on-topic English. It is not a knowledge oracle; it rambles and it is not factually reliable. The interesting part is not the samples, it is a negative result about the piece of the model I was least willing to give up.

Tokens as phasors

Every vocabulary item $v$ owns a learned phase vector $\Phi[v] \in \mathbb{R}^m$, initialised uniform on $(-\pi, \pi]$. Its clean packet is the unit phasor

so every token has identical componentwise magnitude, $|e(v)_j| = 1$, and identity lives entirely in phase. Magnitude is deliberately left free: it becomes the carrier of corruption.

Diffusion as decoherence

The forward process jitters phases with a log-linear noise scale $\theta(t) = 0.05 \cdot 120^{\,t}$:

Each component stays on the unit circle, but its expectation contracts. The Gaussian characteristic function gives the whole story in one line:

Phase information is scrambled, and the degree of scrambling is recorded in the expected magnitude, the textbook signature of decoherence. That envelope $e^{-\theta^2/2}$ is what the schedule is really doing:

This is where the framing earns something. In ordinary discrete diffusion the absorbing [MASK] state is a reserved extra symbol with a learned embedding. Here it is not reserved at all: it is the $\theta \to \infty$ limit of a continuous physical corruption, reached along a path of partially coherent states, and the mask embedding is forced to be the zero vector rather than learned.

The mixer

The denoiser sees $L$ corrupted packets and predicts clean ones at every position. Its blocks contain no attention. Each one advances $i\,\partial_t \psi = (K + V)\psi$ by a single first-order Lie–Trotter split step: apply the kinetic factor, then the potential factor, each exactly, eating the usual error in their commutator.

The kinetic step is diagonal in the sequence-frequency domain, which is what makes it cheap:

with $k = 2\pi\,\mathrm{fftfreq}(L)$ and $(a_1, a_2, a_3)$ learned per complex channel: an advection term, a free-particle term, and an Airy term. Because $|e^{-i\omega_c(k)}| = 1$ this step is exactly unitary: an all-pass filter that can translate, spread and skew packets along the sequence but cannot create or destroy signal energy. The potential step is content-dependent, and is where energy is allowed to leave:

a unitary rotation by $V_r$ plus a strictly contractive absorbing term, since $\mathrm{softplus}(V_i) > 0$ means magnitudes can only shrink.

Here is the kinetic step, running. This is not a video of a simulation; it is the same three lines the mixer executes, in your browser: FFT along the sequence, multiply by the phase plate $e^{-i\omega_c(k)\,\mathrm{d}t}$, inverse FFT. Hue is phase and brightness is amplitude, so the fringes where the two packets overlap are literal interference, and the white trace is the amplitude the Born readout would see.

Watch what the amplitude trace does when the two packets cross: it goes flat and featureless at exactly the moment the hue banding is richest. All of the structure is in phase, and a magnitude-only readout sees none of it. That turns out to be the whole story.

The readout, not the dynamics

The first matched comparisons had the wave model trailing a parameter-matched transformer MDLM by roughly half a bit, and I spent a while assuming the wave dynamics were the problem. They weren't.

The denoiser exposes a complex field $\hat\psi \in \mathbb{C}^m$ per position, and the physics-native way to turn that into token probabilities is the Born rule. Score each token by intensity against its clean packet, $I(v) = |\langle e(v), \hat\psi\rangle|^2$, and take the logit as $\tau\log(I(v)/m + \epsilon) + b_v$. At $\tau = 1$, $b = 0$, $\epsilon \to 0$ the softmax collapses to exactly

token probabilities as relative measurement intensities. It is the most satisfying part of the design, and it was costing real bits.

Swapping it for an ordinary linear head on both quadratures, $\ell = W[\mathrm{Re}\,\hat\psi; \mathrm{Im}\,\hat\psi] + b$, with identical wave dynamics and identical training, moves the model ahead of the transformer. But that comparison is capacity-confounded: the full linear head carries $2mV$ parameters where the codebook-tied Born head carries $mV$. So the load-bearing control is a bottleneck linear head, rank $K = 819$, sized to match the Born head exactly:

text8, BPE-8192, 20k steps, mean of two seeds. Masked-LM bits/token, a conditional slice, not a likelihood.

+0.394 bits at matched parameters. The gap is the functional form of the measurement, not head capacity.

A second control sharpens it. The MLP head at 77.4M does not improve on the full linear head at 73.2M, so nonlinearity buys nothing: the information the Born rule was throwing away is linearly accessible in $\hat\psi$. It was being projected out three ways: phase collapse (the modulus keeps only the magnitude of each inner product), a rank-one PSD constraint (each token scored through $M_v = e(v)e(v)^{\mathsf{H}}$), and norm invariance (the Born softmax is invariant to global rescaling of $\hat\psi$, deleting a degree of freedom outright).

Born-rule readouts have been advocated for language models on physical grounds. At matched capacity, in this setting, that motivation cost 0.394 bits over an ordinary linear map of the same size.

Which property of the mixer actually matters

The symmetric question is whether the Schrödinger form is doing the work. Mostly it isn't. What matters is structured complex spectral mixing:

feyn2 is a structured complex mixer that is not a Schrödinger discretisation, and at char scale it slightly beats the split-step mixer. Removing unitarity or removing complex phase both degrade substantially. So the Schrödinger story is a good source of structure, not a necessary one.

The constraint matters more with scale, which was the surprise. At 57M, removing unitarity costs +0.51 bits despite schrodfree carrying slightly more parameters (58.4M vs 56.9M), roughly twice the char-scale gap. Complex arithmetic alone confers no advantage; the constrained unitary dispersion is what earns the bits, at the scale where the headline comparisons live.

The 548M model

Trained on 9.99B tokens of FineWeb-Edu, sequence length 512, one RTX-5090-week.

The NELBO is the load-bearing number: a proper any-order marginal-likelihood upper bound, computed with two independent MD4 estimators that agree to 0.005 bits, so schedule-invariance holds and the bound is well-estimated. The conditional number is fill-in-the-blank quality: predict 15%-masked tokens given the other 85% as clean context. It is much lower because it is a much easier task. Both are reported because quoting only the second one would be flattering and wrong.

At equal parameters the linear-head wave model matches the transformer baseline, and the wave arm is actually the smaller of the two: 547.9M against 592.7M.

What it can't do

• Not factually reliable. It is 548M parameters. It produces plausible prose, not correct facts, and it drifts.
• Fixed length 512. The dispersion coefficients are learned on the grid $k = 2\pi\,\mathrm{fftfreq}(L)$, which is tied to the training length. This does not transfer across lengths the way attention does, and it is the real structural cost of the approach.
• Weak precise recall. Strong at distributed, statistical context; weak at sharp associative lookup.
• A phrase quirk. It over-uses a few stock phrases picked up from the corpus. A model trait, left unsuppressed.

Prior art it leans on: MDLM (Sahoo et al., 2024), D3PM and SEDD for absorbing discrete diffusion, BD3-LMs for block decoding, LLaDA and Dream for diffusion LLMs at scale, and the split-step Fourier method from nonlinear Schrödinger numerics. The weights and demo are on Hugging Face.