Targeted Model Editing
Suppose I want to make a pretrained model less toxic, remove a visual concept, or change some other narrow behavior. The ideal edit changes that thing and leaves everything else alone.
That second part is the hard one.
Fine-tuning changes parameters that participate in many behaviors. Activation steering is more surgical, but a steering direction can still mix the concept we care about with other differences in the data used to identify it. A direction separating toxic from non-toxic text, for example, may encode topic, tone, vocabulary, or demographics along with toxicity.
So here is a simple idea: identify the concept before learning how to edit it.
Start with counterfactuals
Suppose we can construct pairs of inputs that are as similar as possible except for a target concept C. For an image model, perhaps the same scene with and without a robot. For a language model, perhaps two prompts that differ only in the attribute we want to manipulate.
Run both through the model and take the difference between their activations.
One pair gives us a direction. Many counterfactual pairs give us a collection of directions. If the concept is represented somewhat consistently across contexts, those differences should concentrate in a low-dimensional concept subspace rather than a single universal steering vector.
There are good reasons to expect that a subspace may be more useful than one direction. “Toxicity,” “formality,” or “contains a robot” need not be represented identically in every context. What we want is the small part of activation space that repeatedly changes when C changes.
In practice, I would estimate that space from the leading components of the counterfactual activation differences, ideally after normalizing them relative to the ordinary covariance of activations.
Then constrain the edit
Once we have an estimated concept subspace, constrain the intervention to operate through it.
This can be combined with existing activation-steering objectives. ActAdd, for example, constructs steering vectors from contrasting activations. LinEAS learns small layerwise activation transformations using source and target distributions. The additional constraint here is simply: first estimate the part of representation space associated with C, then restrict learning to those coordinates.
The hope is not that this magically guarantees a clean edit. It gives us a more disciplined place to make the edit.
And it gives us a simple experiment.
Compare:
- an unconstrained activation-steering method; and
- the same basic intervention restricted to the counterfactual concept subspace.
Then measure two things: how much each method changes C, and how much it changes everything we wanted to preserve.
If the constrained intervention achieves the same movement on C with less collateral damage, the geometry bought us something.
Local geometry is not behavioral locality
There is an important catch.
Leaving the orthogonal component of an activation unchanged at one layer does not imply that everything else about the model remains unchanged. Subsequent nonlinear layers can propagate and re-entangle the intervention. A geometrically local edit can have non-local behavioral consequences.
So the claim cannot be “everything outside the subspace is preserved by construction.” What is preserved by construction is much narrower: the intervention at that layer is restricted to a particular set of coordinates.
Whether that produces a targeted behavioral edit is an empirical question.
In fact, this may be the most interesting use of the exercise.
If a concept admits a small subspace and interventions within it change C while mostly preserving unrelated capabilities, that suggests the model represents C in a reasonably separable way.
If every successful intervention causes substantial collateral movement, that tells us something too. The concept may simply be entangled with other representations.