Angular Steering
steerability.algorithms.state_control.angular_steering
args
Angular Steering argument validation.
control
Angular Steering control: rotational activation steering in a learned 2D subspace.
AngularSteering
Bases: InterventionControl
Angular Steering.
Rotates the hidden state within a per-layer 2D plane spanned by a feature axis (row 0 of the
steering vector) and a companion axis (row 1), leaving the orthogonal complement (the other
d_model - 2 directions) untouched. Because a 2D rotation is orthogonal, the intervention is
norm-preserving by construction and offers continuous control via a single angle.
The method operates in two phases:
-
Offline plane fitting: A per-layer feature axis is estimated via difference-in-means over contrastive data, and a single global companion axis is taken as the first principal component across the stacked per-layer feature directions. Gram-Schmidt yields an orthonormal
(b1, b2)per layer. A precomputed[2, H]-per-layer plane may be supplied directly instead. -
Online rotation: A
forward_pre_hookon each layer's residual-stream normalization sub-modules (input_layernormandpre_feedforward_layernormon Gemma,input_layernormandpost_attention_layernormon Llama/Mistral/Qwen,ln_1/ln_2on GPT-2) rotates the residual stream entering the norm to the target angle (mode="target") or by the angle (mode="offset"). Vector addition and directional ablation are special cases of this rotation. The adaptive variant rotates only tokens already positively aligned with the feature axis, improving coherence on smaller models.
The control is declarative: _configure maps the validated args onto one Intervention
over the artifact's covered layers, at the "norm_input" site by default or the decoder
layer output when intervention_point="layer_output" (the only placement with an
intervention-spec form, since the mid-layer boundary exists only inside the in-process
forward pass).
Reference:
- "Angular Steering: Behavior Control via Rotation in Activation Space" Hieu M. Vu, Tan M. Nguyen https://arxiv.org/abs/2510.26243
Source code in steerability/algorithms/state_control/angular_steering/control.py
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 | |
RUNTIME_KWARGS_SCHEMA = []
class-attribute
instance-attribute
_gate
property
The first intervention's gate (None before steer() or when unconditional).
_steering_vector
property
The bound steering plane as a SteeringVector view (None before steer()).
_template = ()
class-attribute
instance-attribute
_transform
property
writable
The first intervention's transform (None before steer()).
Assignment replaces the transform on the first intervention, so a wrapped or instrumented transform takes effect in subsequently built hooks.
args = self.Args.validate(*args, **kwargs)
instance-attribute
enabled = True
class-attribute
instance-attribute
hook_only_hint = "norm-input rotation has no intervention-spec form; set intervention_point='layer_output' or run on the huggingface backend"
class-attribute
instance-attribute
interventions = ()
class-attribute
instance-attribute
supports_batching = True
class-attribute
instance-attribute
tokenizer = None
class-attribute
instance-attribute
__init__(*args, **kwargs)
Source code in steerability/algorithms/core/base_control.py
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 | |
_configure()
Post-construction hook, called after Args fields are mirrored onto the instance.
Driver presets override this to map their mirrored args onto the fields their generic base
reads, so subclasses never bypass a parent __init__. Default no-op.
Source code in steerability/algorithms/state_control/angular_steering/control.py
63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | |
_fit_sources()
Yield the template's fit sources: unbound sources carrying an artifact_class
whose resolution does model-side work (access above ModelAccess.FACTS).
Source code in steerability/algorithms/state_control/base.py
377 378 379 380 381 382 383 384 385 | |
_resolve_module_layout(model=None)
The module-path layout, resolved from the module tree on first use.
Source code in steerability/algorithms/state_control/base.py
257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 | |
_unbound_sources()
Yield each unbound template element's source (or undeclared factory slot).
Yields the transform sources of unbound transform elements, factory transform slots
themselves (which declare their own access or default to the live model), and
unresolved gate sources, in template order.
Source code in steerability/algorithms/state_control/base.py
342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 | |
cleanup()
Release resources allocated during steer().
Override in subclasses that allocate GPU memory or other resources during steering. Default no-op.
Source code in steerability/algorithms/core/base_control.py
195 196 197 198 199 200 201 | |
clone_for_call(seed=None)
A per-call clone whose interventions carry independent gate state.
Gates are deep-copied with one shared memo across the control's interventions, so a gate instance shared by several interventions stays shared inside the clone while being isolated from the original and from sibling clones. Transforms, scorers, and steer-time artifacts stay shared.
Source code in steerability/algorithms/state_control/base.py
517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 | |
export_intervention_spec(runtime_kwargs=None)
The control's InterventionSpec, lowered from the bound interventions, or None.
Must be called after steer(). Returns None when the configuration has no wire form.
Source code in steerability/algorithms/state_control/base.py
320 321 322 323 324 325 326 327 328 329 330 331 332 | |
export_state()
Bound steering artifacts, keyed by intervention position.
For intervention i, the core transform's artifact exports as
"intervention_{i}/transform" (a SteeringVector view), each wrapper's own artifact
as "intervention_{i}/modifier_{j}", and the resolved gate as
"intervention_{i}/gate" when the template's gate slot held a source. Must be called
after steer().
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
Mapping from logical name to artifact value. |
Source code in steerability/algorithms/state_control/base.py
402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 | |
fit_identity()
The template's fit sources in template order, each in its encodable form, or None when the template declares no fits.
Source code in steerability/algorithms/state_control/base.py
394 395 396 397 398 399 400 | |
frozen_form(state)
One activation_adapter entry per bound intervention, in intervention order.
Each entry's args mirror the intervention: the bound transform, the resolved behavior
layers, the hook point, the resolved gate, and the token scope. Must be called after
steer().
Returns:
| Type | Description |
|---|---|
list[tuple[str, dict[str, Any]]]
|
List of |
Raises:
| Type | Description |
|---|---|
NotFreezableError
|
If the control is unsteered, an intervention hooks the
|
Source code in steerability/algorithms/state_control/base.py
434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 | |
get_hooks(input_ids, runtime_kwargs=None, attention_mask=None, **kwargs)
Compile the bound interventions to hooks for the current generation.
Delegates to build_hooks: a fresh hook runtime is created, gates reset to the
logical batch, and one behavior hook is emitted per (intervention, layer).
Generation-time context arrives through **kwargs; model is consulted to resolve
hook module names when steering ran without a live model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_ids
|
Tensor | dict
|
Prompt token ids of shape |
required |
runtime_kwargs
|
dict | None
|
Unused. |
None
|
attention_mask
|
Tensor | None
|
The prompt attention mask matching |
None
|
Returns:
| Type | Description |
|---|---|
dict[str, list[HookSpec]]
|
Hook specifications with |
Source code in steerability/algorithms/state_control/base.py
273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 | |
requirements()
Backend requirements derived from the declared interventions, per phase.
Generate offers the intervention-spec alternative whenever every component of every intervention has a wire form; hook-only configurations require the in-process backend. Score is in-process: remote prompt-logprob scoring anchors token scopes at the request's prompt end (the end of the prompt-plus-reference concatenation), which would silently unanchor prompt-relative interventions.
Source code in steerability/algorithms/state_control/base.py
479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 | |
steer(model=None, tokenizer=None, session=None, **kwargs)
Bind the intervention template against the model (or the session's layout).
Structural facts come from the steering session's layout when a session is given, so a
fully concrete template (precomputed vectors, manual thresholds) binds with
model=None. Templates carrying sources (fits, searches) resolve them here.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
model
|
PreTrainedModel | None
|
The base language model, or None for concrete templates bound against a session layout. |
None
|
tokenizer
|
PreTrainedTokenizerBase | None
|
Tokenizer used when fitting sources. |
None
|
session
|
SteeringSession | None
|
|
None
|
Returns:
| Type | Description |
|---|---|
PreTrainedModel | None
|
The input model, unchanged. |
Source code in steerability/algorithms/state_control/base.py
198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 | |
steer_access()
The model access the template's steer step requires, folded over its sources.
A fully bound template requires ModelAccess.FACTS, since pure layer selectors
resolve from structural facts available on any session. Otherwise the strongest
declared source access wins; a source or factory slot without an access declaration
builds against the live model, so it counts as ModelAccess.MODULE.
Source code in steerability/algorithms/state_control/base.py
364 365 366 367 368 369 370 371 372 373 374 375 | |
steer_fits()
The template's fit artifacts, i.e. every unbound source carrying an
artifact_class, as (artifact, artifact_class) pairs in template order.
Source code in steerability/algorithms/state_control/base.py
387 388 389 390 391 392 | |
wire_kinds()
The combined wire kinds of the bound interventions (or the template before
steer()), or None when any intervention is hook-only.
Source code in steerability/algorithms/state_control/base.py
334 335 336 337 338 339 340 | |