Weight-space programming: editing Qwen3.5 into a top Pac-Man decision model

Weight-space programming means changing what a model does by writing its weights directly, each change computed from recorded activations rather than learned by gradient descent. This is the story of doing it for real: Jevlar, said like the Swedish jävlar [yev-lar], the word you shout when a ghost takes your last life, a Pac-Man decision model edited into Qwen3.5, from a first score of 1327 to 3604 over 100 games and the top of Opper's JevMan leaderboard.

The JevMan benchmark and the rules

JevMan is Opper's Pac-Man benchmark for decision models: models that answer a multiple-choice question in one pass, by reflex rather than by reasoning, in real time. The game sends a state and a question at every junction ("up, left or right?") and whenever a ghost comes down the corridor ("keep going or turn back?"). The model returns probabilities over the options, the game draws one and moves on. There is no generation and no chain of thought, and a late answer is a wrong answer: Pac-Man stands at the junction while the ghosts keep moving.

Jevlar is our entry, built under rules borrowed from the Sparks of Weight-Space Programming experiments. No gradient descent, the thing people usually mean by training. The model's input and output layers stay untouched, and no shortcuts such as scaling a whole matrix. Every edit is computed in one step from recorded activations and written into the weights: an attention head told what to look at, a neuron told what number to write, a linear read-out folded into a weight matrix.

The practice data comes from the benchmark's own simulator, played by its greedy rule, and benchmark games are never used for fitting. Because the read-outs are fitted on simulator positions, the entry carries the benchmark's "trained" label.

Two agents, one repository

The weights were edited by two coding agents under one person's direction, in the same checkout, on the same rules. Claude's Fable 5.1 built Jevlar: the circuits, the regime maps, the edit-and-play loop and the Modal containers that play the benchmark next to the model. OpenAI's Codex agent, Astra, ran a competing build with its own notebooks in the same tree: a last-layer correction fitted by ridge regression (2111), a mixture of four specialists, one per game situation (2304), experiments with thinking tokens and a full-board view, and at the end a replay of our sets at three answer speeds.

The two shared a GPU lock on the laptop so their passes never overlapped, a one-run-per-container rule on Modal after two concurrent runs slowed each other to 540 ms an answer, and a rule that each commits only its own paths. They read each other's notebooks rather than talking; a chat relay between them was tried for a day and dropped. Ideas went both ways: Astra's readout correction became the last layer of Jevlar, re-fitted on the 4B, and Fable's speed finding sent Astra off to replay hc1 with added delays, which is why its control run of our set sits in the pooled numbers.

The human in the loop set the budgets and the direction: stay on the small model, try something radical when it plateaued, find out whether we were dying from being slow, play the 100 games. Passes ran on the laptop, an M3 Max at 0.4 s a position for the 4B. Games ran in Modal containers on A10Gs and H100s, about $50 of GPU time in all.

The starting line

The target was the top of the leaderboard: the greedy rule at 4450, Typesafe's jev 1.14 at 3597, the trained ModernBERT-jev at 3180 and its previous version, jev 1.13, at 2750. The greedy rule is the hand-written search that also plays the practice games, so it is the teacher and the ceiling at once.

The base models start far below that. Qwen3.5-0.8B with our first four edited attention heads scored 1327, and the unedited 2B 1277. The unedited 4B scores 2148 to 2201 once the prompt lists the options flagged as dangerous first, which already beats every edited 0.8B set we had. Every score in this post is a mean over 24 games unless it says otherwise, and a 24-game mean of the same weights moves by about 450 from run to run, so the story is told in pooled runs where it can be.

The climb

lab notebooks · 24-game means, except fast serving (120 games pooled) and the 100-game run

The first three steps are the 0.8B: four edited heads that read the dangerous options (v1), the first ghost-hunting circuit (v3) and its reach ceiling (v5c). Then a plateau: everything we added to the 0.8B landed between 1800 and 2150. The radical step was to list the dangerous options first in the prompt, which unlocked the 4B base at 2175 with no edit at all. Porting the hunting circuit to the 4B, the recipe re-fitted rather than the weights copied, gave 2750, the refitted readout correction 2911, serving at under 100 ms about 3300 pooled, and the 100-game run 3604. The 0.8B's own late jump to 3110 with v7 is told below.

What the weight edits are

Jevlar changes 0.69% of Qwen3.5-4B's weights, in four places, and every change is computed, not trained.

• A copy head (layer 15). At every number of steps in the prompt it looks back a few words to see what the number belongs to and, if that is a frightened ghost, stamps a tag on it. Nothing in the base model tags those numbers.
• A number neuron (layer 15). One neuron that fires only on tagged numbers and writes 20 − N where the next part can read it: positive for a ghost within reach, negative beyond it. The ceiling of 20 steps matters; a version with 40 chased ghosts it could never catch.
• A direction head and a fright head (layers 15 and 19). The first notes which way each option points. The second, active only while the ghosts are frightened, compares the options, picks the one with the best reach and reads off its direction. Together they make the model hunt: ghosts eaten a game went from 0.8 to 2.3.
• A readout correction (layer 31). A linear correction, fitted on 1536 practice positions, that nudges the final answer toward what the greedy rule would pick, written into the last layer's weights. It lifts agreement with the greedy rule from 81% to 90% on games the fit never saw.

the hunting circuit of Jevlar on Qwen3.5-4B · 5 edits, 2 positions

The top row runs at every number in the prompt and leaves a tagged, signed reach behind each frightened ghost; the bottom row runs once, at the answer, and turns the best reach into a direction.

Everything ships as the unchanged base model plus one small file of edited weights, loaded at start. The 0.8B's edits follow the same pattern at layers 11, 15 and 22.

The recipe transfers to a bigger model

The whole circuit was designed on the 0.8B and transferred to the 4B as a recipe: the same four parts, re-fitted on the larger model's own recordings in three forward passes, placed at the layers where the 4B carries the same signals (15 and 19 instead of 11 and 15). The port took an afternoon and lifted the unedited 4B from 2175 to 2750 on its first run, and the readout correction moved over the same way with the same script. Nothing about the method is tied to one model size. Re-fitting meant re-running the same three recipes on the bigger model's recordings: what the copy head should look at, what number the neuron should write, and how the two heads' outputs map into the bigger model's directions. The design decisions, the tag, the 20-step ceiling, the two-position layout, all carried over unchanged.

The same weights, 700 points apart

The biggest single jump came from serving, not from the weights. At 193 ms an answer, inside a container on an A10G, hc1 scored 2911 over 96 games. At 54 to 72 ms on an H100, the same file scored 3290 over 120 games and 3604 over the 100-game run.

The mechanism is the escape question. When a ghost comes down the corridor the benchmark asks "keep going or turn back?" and re-asks as the situation changes. An answer that arrives after its question was replaced is logged as stale and never used. At 193 ms Jevlar got 6 escape answers a game in on time and 16 arrived stale; at 58 ms it got 33 in and 12 stale. Deaths with a ghost ahead in the corridor fell from 29 to 47 of 72 to 7, and ghosts eaten rose from 2.0 to 3.1 a game.

ten runs of edits/jevman-4b-hc1 inside Modal containers, ours and Codex's · 24 games each, one of 100

The scatter inside each band is the run-to-run noise of 24 games; the step between the bands is the speed.

Two things made the speed. Part of the model had been running on slow fallback code in the container; installing the proper GPU kernels cut the answer from 193 to 119 ms on the same hardware. The H100 then took it to 54 ms. Speed only pays below about 100 ms: at 119 ms the score was unchanged. The small model gains nothing from speed at all, because its deaths are decisions at junctions, not late answers.

Where the decision forms, and the small model's 1000 points

Three forward passes per model recorded what every layer computes at the moment of the answer, for 1536 labelled practice positions. Two facts came out. Which situation the game is in (hunting, survival, collection, escape) can be read off the model's internal state from layer 2 on, with 99% accuracy, because the prompt says so. And the right move can be read off that state from the middle of the network onward: in the 0.8B a simple linear probe matches the greedy rule on 80% of unseen positions from layer 16 on, while the model's own output manages 63%. The model knows the answer halfway through and loses a third of it on the way out.

regime maps of both models · 1536 practice positions, validation on the last 28% of games

The gap between a model's curve and its dashed line is what a read-out can recover: 17 points on the 0.8B, nothing on the 4B.

That gap is an edit. We take the probe that reads the right move from a middle layer and build it into four of that layer's least used neurons, so that they write the move back into the model's state where the output can use it. Four neurons, one script, no training. In the loop (edit, play eight container games, repeat) the layer-22 version at half strength went from v5c's 2100 to 3028 on its first eight games, and held 3110 ± 187 over 48. That is Jevlar v7, the 0.8B at 55 ms, 0.73% of its weights changed.

The 4B did not have the gap. The right move is 89% readable at layer 24 and its last-layer correction already delivers 90%, and the same neurons moved its score sideways in 24-game runs. The map told us which model had room before we spent the games.

What did not work

Most of the effort went into things that failed, and the failures shaped the method.

• Matching the teacher on test positions is a weak proxy for points. Sets with 57% to 77% agreement all scored 1800 to 2150 on the 0.8B. The points came from two specific behaviours, hunting and surviving longer, not from matching the teacher more often.
• The existing parts are already tuned. Turning the hunting head down to 3 or up to 8 from its 5 halved the score; the heads that read danger, at other settings, landed exactly where they started, confirmed over 8 games each.
• Four specialists lost to one correction. Four separate corrections, one per game situation with a learned switch between them, Codex's idea, lift the 0.8B by 300 points but lose to a single correction on the 4B in every situation, even when the switch is told the right situation. Splitting 1200 fitting positions four ways costs more than the specialisation gains.
• Heavier is worse. A strong last-layer correction on the 0.8B matched the teacher on 82% of test positions and still lost 250 points in games, because it overrode the model so hard that it turned a quarter of its own mistakes from right to wrong. The neurons that worked write gently, at half the designed strength.
• Single runs mislead. The 8-game probes that read 4220 and 3940 came back as 2914 and 3216 over 24 games. Nothing was believed until it held over 48 games, and the speed finding was reported as 3600 before Codex's extra runs pulled the pooled figure to 3290.

A competing build by Codex, an OpenAI agent working in the same repository on the same rules, reached 2111 with its own last-layer correction and 2304 with the four specialists. Its best weight-space idea, the readout correction, became part of Jevlar; its last three experiments replayed our sets as controls.

Where it stands

Jevlar's 100-game run on an H100 scores 3604 ± 314, above every model entry on the board and below only the greedy rule it learned from.

Error bars are two standard errors over the games, the submission checker's convention, and the checker validates every recording before it is counted. The edits, the notebook of every pass and the 100 recordings are published with the entry.