SFT vs. RL: Weight Spectra & ISO Explained

Imagine you are mentoring a junior engineer on how to write production SQL. There are two completely different ways you can teach them:

Option one: you sit down next to them, show them 50 perfect query examples, and have them type them out until the syntax becomes muscle memory. Option two: you give them a sandbox database with unit tests, tell them what data needs to come out, and let them write whatever queries they want—scoring them whenever the tests pass and letting them learn from their own mistakes.

The first approach is supervised fine-tuning (SFT). The second is reinforcement learning with verifiable rewards (RLVR).

Lately, AI discussions have tried to flatten this distinction into a clean catchphrase:“SFT injects knowledge, while RL teaches reasoning.”It sounds neat, but in practice, reality is much messier. SFT on detailed worked examples can teach procedural reasoning, and an RL agent interacting with an environment constantly acquires new knowledge through tool observations and trial-and-error.

The real puzzle this article explores: Inside the neural network, how can a weight matrix radically transform what it does while keeping its underlying singular values almost unchanged? Understanding this geometry demystifies the recent ISO paper (Zhu et al., 2026)—and helps separate a genuinely clever optimization technique from broad philosophical claims about where knowledge lives.

1. A Weight Matrix Changes Direction and Scale

Strip away the hype, and a transformer is fundamentally a massive collection of linear projections. Each layer takes an incoming vector of numbers, multiplies it by a weight matrix W, and emits an output vector. Singular Value Decomposition (SVD) breaks that multiplication down into three intuitive steps:

y = Wx = U · Σ · Vᵀx

Read the operations from right to left, starting with your input x:

V^T: express the input in a new coordinate system. This finds the natural input axes along which the matrix acts independently.
Σ: stretch or shrink along those axes. The diagonal values are nonnegative numbers called singular values. Together, they form the matrix’s spectrum.
U: rotate the result into the output space. These directions and the input directions are the two singular frames.

In two dimensions, think of a rubber circle being stretched into an ellipse. The singular values (Σ) decide the length and width of the ellipse. The frames (V^T and U) decide which direction gets stretched and where the stretched shape points.

Why the Same Spectrum Can Produce a Completely Different Output

Here is the geometric intuition that changes how you think about weights: take a matrix that doubles the horizontal coordinate and leaves the vertical coordinate alone. It sends the point (1, 0) to (2, 0). Now simply rotate its output frame by 90 degrees: the exact same input (1, 0) now lands at (0, 2).

Both matrices have the exact same singular values (2 and 1), yet their outputs are completely different.Keeping the spectrum fixed leaves massive geometric freedom to change what the network actually computes.

A scale is not a fact. Singular values describe matrix geometry, not human concepts. You cannot point to a singular value and say “that is where SQL joins live.” A model’s intelligence emerges from the complex dance between thousands of interacting matrices, activations, and attention patterns.

2. Interactive 3D: Change the Frames, Keep the Stretches

Use the 3D visualizer below to see this in action. Start with ISO and drag the progress slider: watch how the output vector changes while the stretch factors remain rock-solid. Step throughV^T, Σ, and U to watch the geometry unfold.

INTERACTIVE LINEAR ALGEBRA · ILLUSTRATIVE VALUES

Same stretches. A different transformation.

Directions change while every scale stays exactly at its starting value. ISO explicitly constrains the spectrum during optimization.

4. Apply U

Orient the result in the output space. The ellipsoid’s axis lengths stay the same; its direction changes.

Illustrative training progress 100%

Move the slider from 0 to 100. In ISO, the white arrow moves even though the three scales below stay fixed. Camera controls only change your view.

Stretch factors · Σ

σ₁

2.300

σ₂

1.500

σ₃

0.800

Starting values: 2.300 / 1.500 / 0.800

A 3 × 3 teaching example, not model weights or measured training trajectories. Colored curves track input directions; the faint sphere is the input reference. A transformer contains many much larger, often rectangular matrices.

3. What the ISO Paper Actually Shows

In ISO: An RLVR-Native Optimization Stack, Zhu et al. made a fascinating empirical observation during RLVR training runs, which they turned into a constrained optimization method:

Observation: Small Spectral Drift. In the RLVR runs they studied, the learned weight matrices remained surprisingly close to their initial singular values. By contrast, an illustrative SFT run rewrote the spectrum significantly. (Note: this is an empirical observation in specific setups, not a universal law of nature).
The Reset Experiment: The authors took RL-trained checkpoints, forcibly replaced their singular values with the initial pre-RL numbers, and kept only the newly rotated frames. Most of the reasoning gains survived! That is strong evidence that the frame rotations were doing the heavy lifting.
The Training Constraint: They asked: what if we freeze Σ from step zero and only train the frames? In their tests, training only the frames worked well, while training only Σ with frozen frames gave minimal gains.

The authors dubbed this functional preservation spectral inheritance.

Observation vs. Hard Constraint: Ordinary RLVR is naturally nearly isospectral in these experiments; ISO enforces it as an intentional constraint. It is a powerful parameterization for optimizers, but it does not prove that reasoning is purely a rotation or that RL cannot acquire knowledge.

4. Turning Geometry into an Optimizer: ISO in Practice

ISO-Optimizer keeps the initial singular values Σ₀ locked in place, updating only the frame variables using AdamW or Muon, followed by an orthonormalization projection step:

W(t) = U(t) · Σ₀ · V(t)ᵀ

The headline paper result on Qwen3-8B-Base on math reasoning:

Qwen3-8B-Base · Reported Math Accuracy (Average@16, Zhu et al.)
Optimizer	Training Steps	Accuracy
Standard AdamW	270	0.495
ISO-AdamW	100	0.495
ISO-AdamW	210	0.509

Reaching the same benchmark in 100 steps instead of 270 sounds like an instant 2.7x speedup.However, as every systems engineer knows, fewer steps does not always mean less clock time or lower cloud bills.The projection step adds mathematical overhead (~7% per step on 4B models), and ISO rollouts tended to be longer, which changes inference cost during RL exploration. Always measure actual wallclock time and dollar cost to target score.

5. Practical Guidance: Which Tool for Which Problem?

When you are sitting in front of your cluster deciding how to improve a model for your team:

What the Model Is Doing Wrong	Best Engineering Move	What to Watch Out For
It mangles output syntax or ignores tool schemas.	SFT on clean, high-quality expert demonstrations.	Overfitting and format repetition.
Its syntax is fine, but it picks flawed logical paths or incorrect SQL joins.	RLVR with deterministic test cases and verifiers.	Reward hacking loopholes where the model cheats the test.
It simply does not know your proprietary database schema.	Inject context via RAG or system prompts first.	Don’t train weights on facts that change every week.
Your RLVR pipeline works, but step convergence is sluggish.	Experiment with frame-focused optimizers (ISO).	Measure real wallclock time and memory overhead, not just step counts.

The most reliable post-training pipelines we see in practice use a pragmatic one-two punch: use SFT to show the model how to hold the tools, then use RLVR to teach it how to solve hard problems on its own.