Steering Controls

Note

This document provides the current list of steering controls. To add your own steering control/method, please refer to the tutorial. For a better understanding of how steering methods can be composed, please see the high-level outline on steering pipelines.

We structure steering methods across four categories of control, loosely defined as:

• input: edits the prompt
• structural: edits the weights/architecture
• state: edits the (hidden) states
• output: edits the decoding/sampling process

The category of a given steering method is dictated by what aspect of the model the method influences. We define each category of control below.

Input control

Baseline model: \(y \sim p_\theta(x)\)

Steered model: \(y \sim p_\theta(\sigma(x))\)

Input control methods describe algorithms that manipulate the input/prompt to guide model behavior. They do not change the model itself. This is enabled in the toolkit through a prompt adapter \(\sigma(x)\) applied to the original prompt \(x\). A pipeline may contain several input controls, which compose in controls-list order, each receiving the previous control's output.

For a control method to be deemed an input control method, it must satisfy the following requirements:

• Control: Method only influences the prompt supplied to the model. It does not change the model's internals (parameters/states/logits).

• Persistence: All changes are temporary. Removing the prompt adapter \(\sigma()\) yields the base model.

• Access: Implemented without requiring access to the model's internals, e.g., hidden states.

Some examples of input control methods are few-shot prompting, reasoning guidance (like CoT, ToT, GoT, self-consistency), automatic prompting methods, and prompt routing. The toolkit implements:

• FewShot (API reference, notebook)
  • Description: pool- or runtime-supplied few-shot examples with a pluggable selector.
  • Backends: HF, vLLM.
• PRewrite (API reference, notebook)
  • Description: RL-trained instruction rewriter (Kong et al. 2024) supporting a greedy "inference" strategy and a best-of-K "search" strategy. The rewriter can optionally be trained with GRPO using a scorer-in-the-loop reward (apply the rewrite with the frozen task model over a dev set and score each response with a per-row SampleScorer, the paper's reward).
  • Backends: HF, vLLM.
• CPO (API reference, notebook)
  • Description: causal prompt optimization (Chen et al. 2026), i.e., offline causal reward training (Double ML over PCA-reduced embeddings) plus per-query tree search.
  • Backends: HF, vLLM (requires prompt_lm). Without prompt_lm the pipeline's loaded model is bound as the proposer, which is HF-only.
• GEPA (API reference, notebook)
  • Description: reflective genetic prompt evolution (Agrawal et al. 2025), single-module variant.
  • Backends: HF, vLLM.
• SystemPrompt (API reference, notebook)
  • Description: sets or merges the leading system message of a chat, prepending to, appending to, or replacing it (the default is to prepend ahead of an existing system prompt), always producing exactly one system message.
  • Backends: HF, vLLM.
• UserPrefix (API reference, used in notebook)
  • Description: prepends a fixed text marker to a user turn (the last user turn by default, or the first or all user turns), with the token stream as a fallback for non-chat input.
  • Backends: HF, vLLM.

The few-shot retriever from Rubin et al. 2021 (EPR) is provided as a BaseSelector that slots into FewShot rather than as a separate control. See few_shot.selectors.epr.

Reusable building blocks shared across these methods (memory containers, formatters, scorers, proposers, selectors, Pareto / rollout-budget utilities) are located in input_control.common.

Structural control

Baseline model: \(y \sim p_\theta(x)\)

Steered model: \(y \sim p_{\theta'}(x)\)

Structural control methods alter the model's parameters or architecture to steer its behavior. These methods usually allow for more aggressive changes to the model (compared to input control methods). Structural controls are implemented via fine-tuning, adapter layers, or architectural modifications (e.g., merging) to yield an updated set of weights \(\theta'\). A pipeline may contain several structural controls, and steer() threads the model through them in controls-list order.

Structural control methods satisfy the following requirements:

• Control: Produces a new or modified set of weights \(\theta'\) or extends the network with additional modules/layers.

• Persistence: Changes are persistent and are stored in the checkpoint. Reverting requires reloading or undoing the weight edit.

• Access: Implementation requires access to parameters and (typically) gradient flows.

Examples of structural control methods are fine-tuning methods (full, parameter efficient), soft prompting (prefix tuning, p-tuning), and model merging. Many of the structural control methods in the toolkit are implemented as wrappers around existing libraries. The toolkit implements:

• LoadCheckpoint (API reference)
  • Description: installs a saved full-weights checkpoint as the pipeline model, the frozen form of trained structural controls in a .spipe bundle.
  • Backends: HF, vLLM (the checkpoint is served).
• LoadLoRA (API reference)
  • Description: attaches a saved LoRA adapter to the pipeline model (optionally merging it into the base weights), verifying the adapter's recorded base model. This is the frozen form of adapter-producing structural controls in a .spipe bundle.
  • Backends: HF, vLLM (the adapter is served).
• MergeKit (API reference, notebook)
  • Description: model merging via MergeKit¹, combining multiple checkpoints with strategies such as linear interpolation, SLERP, and TIES from a YAML/dict config.
  • Backends: HF, vLLM (the merged checkpoint is served).
• TRL (API reference, notebook)
  • Description: weight-level training via Hugging Face TRL², exposing SFT, DPO, APO, PPO, and GRPO trainers, with optional LoRA/PEFT and a post-training merge. Since training_args is forwarded verbatim to the installed TRL config, a key the config does not declare raises an error at control construction.
  • Backends: HF, vLLM (serves the steer-time artifact, a checkpoint or LoRA adapter, and requires a configured output directory).

State control

Baseline model: \(y \sim p_\theta(x)\)

Steered model: \(y \sim p_{\theta}^a(x)\)

State control methods modify the model's internal/hidden states (e.g., activations, attentions) at inference time. These methods are implemented by defining hooks that are inserted/registered into the model to manipulate internal variables during the forward pass.

State control methods satisfy the following requirements:

• Control: Writes to (augments) the model's internal/hidden states. Model weights remain fixed.

• Persistence: Changes are temporary. Behavior reverts to baseline once hooks are removed.

• Access: Requires access to internal states (to define hooks).

Some examples of state control methods are activation addition/steering, attention steering, and representation patching. The toolkit implements:

• ActAdd (API reference, notebook)
  • Description: activation addition³, adding a positional steering vector from a single contrast pair to the residual stream at one layer.
  • Backends: HF (positional injection has no intervention-spec form).
• ActivationAdapter (API reference, notebook)
  • Description: the composable activation-steering atom, wiring together the shared common components (a transform that contains its own artifact, a selector, a gate, and a token scope) so that a recipe is assembled without writing a new control class.
  • Backends: HF, vLLM (kind-conditional, i.e., the configured transform, modifier chain, and gate readout/rule must all have wire forms, and a CallableReadout gate is HF-only).
• AngularSteering (API reference, notebook)
  • Description: angular steering⁴, rotating the hidden state within a per-layer 2D plane (feature axis + companion axis) to a target angle while leaving the orthogonal complement untouched. It is norm-preserving by construction, and vector addition and directional ablation are special cases.
  • Backends: HF, vLLM (intervention_point="layer_output" only, since the default norm-input placement is HF-only).
• CAA (API reference, notebook)
  • Description: contrastive activation addition⁵, adding a learned mean-difference direction to the residual stream at a single layer.
  • Backends: HF, vLLM (norm-preserving configurations included).
• CAST (API reference, notebook)
  • Description: conditional activation steering⁶, applying behavior steering only when a learned condition direction crosses a threshold. The applied behavior transform is pluggable (additive by default, or any BaseTransform via behavior_transform, e.g., directional ablation for conditional abliteration).
  • Backends: HF, vLLM (with the default additive behavior transform, while a custom behavior_transform follows that transform's wire form).
• DirectionalAblation (API reference, notebook)
  • Description: directional ablation / abliteration⁷, projecting a learned feature direction (or subspace) out of the residual stream at masked positions, with a graded ablation strength.
  • Backends: HF, vLLM (single direction at full strength, K = 1 and alpha = 1, while graded and subspace ablation are HF-only).
• ITI (API reference, notebook)
  • Description: inference-time intervention⁸, shifting activations at a sparse set of probe-selected attention heads during generation.
  • Backends: HF, vLLM (tensor_parallel_size == 1, norm-preserving configurations are HF-only, and fitting from data runs on the staged model).
• PASTA (API reference, notebook)
  • Description: post-hoc attention steering⁹, rescaling attention to targeted prompt substrings at selected layers and heads. The head_config argument takes a dict or list of layers and heads, or a HeadProfile recipe that runs the paper's head-profiling stage as a steer-time fit on the loaded model (scoring each candidate head by its paired lift over an unsteered baseline) and freezes the resolved head map.
  • Backends: HF with attn_implementation "eager" or "sdpa" (attention-map writes have no engine form).

Reusable building blocks shared across the residual-stream methods (estimators, gating, selectors, transforms, steering vectors, hook utilities) are located in state_control.common.

Most state controls in the toolkit are declarative. A control states its edit once, as a tuple of interventions, where each intervention specifies the layers to edit, a transform (e.g., adding a direction or projecting one out), a token scope (which positions receive the edit), and optionally a gate. The toolkit compiles this statement for whichever backend runs it, i.e., to torch hooks in process and to an intervention spec for engines that host activation edits through the vLLM-Hook plugin. A configuration whose components all have a serialized form therefore generates on vLLM without any control-specific code, and one that does not stays in process. The pipeline's check() reports which, with a verdict that identifies the gap and the fix.

A gate makes an intervention conditional. It reads hidden states at chosen layers, reduces each pooled state to a per-prompt value (e.g., an affine score or a cosine similarity against a condition direction), and applies a decision rule (e.g., a summed score against a calibrated bias, or per-layer thresholds). The decision is made on the prompt, applies for the whole generation, and is taken independently per row of a batch. An unconditional intervention has no gate.

ActivationAdapter is the general-purpose form of a declarative control. Each adapter steers a single behavior (one transform chain, one gate, and one token scope), and steering with several behaviors is several adapters listed together in a pipeline's controls, applied in list order. Adapters can share one gate instance for joint conditioning, and a fitted Probe can gate an adapter through Probe.as_gate(). Position-scoped and gated controls compose with multi-call decoding drivers (e.g., segment search) and with step-level controls that score candidates through the pipeline's own model.

State controls locate the decoder layers through a model layout, which is resolved automatically for text-only decoder models (Llama, Mistral, Qwen, and Gemma), for composite multimodal wrappers loaded under AutoModelForCausalLM (Gemma 3/4 and Qwen3.5), and for GPT-2. Hybrid architectures that interleave attention layers with another token mixer (Qwen3.5 and Qwen3-Next) are supported by the residual-stream controls and by hidden-state capture, while controls that act on attention (PASTA and o_proj-site interventions) are restricted to the attention layers, and ITI does not support them. A multimodal checkpoint is steered on its text decoder under text-only prompting, and images and audio are out of scope. A state control listed after an unmerged LoRA adapter steers the adapted model. For an architecture not on this list, register a detector with register_layout_detector (from steerability.algorithms.core.internals).

Output control

Baseline model: \(y \sim p_\theta(x)\)

Steered model: \(y \sim d(p_{\theta})(x)\)

Output control methods modify model outputs or constrain/transform what leaves the decoder. The base distribution \(p_\theta\) is left intact, and only the path through the distribution changes.

Output control methods satisfy the following requirements:

• Control: Replaces or constrains the decoding operator. No prompts, hidden states, or weights are altered.

• Persistence: Changes are temporary. Behavior is restored once decoding control is removed.

• Access: Requires access to logits, token-probabilities, and possibly hidden states (depending on the method).

Examples of output control methods are sampling/search strategies, weighted decoding, and reward-augmented decoding. Output controls participate in decoding in one of two ways. A step-level control supplies logits processors and/or stopping criteria (via get_logits_processors and get_stopping_criteria), which the pipeline composes in controls-list order. Step-level controls therefore compose with each other and with a decoding driver. A decoding driver subclasses DecodingDriver and owns the decode loop (decode(...)), applying the composed processors and stopping criteria in every forward pass it issues. Since the loop does not compose, a pipeline admits at most one enabled driver, and with none, decoding defaults to the model's own generate.

The toolkit implements the following step-level controls:

• RAD (API reference, notebook)
  • Description: reward-augmented decoding¹⁰, scoring the top-k candidate tokens with an AutoModelForSequenceClassification reward model and shifting their logits by beta * reward. When the reward model is decoder-only and shares the base model's vocabulary it caches the reward-model prefix activations across steps (the paper's efficient path), and otherwise scores each step statelessly. A value_trace list passed via runtime_kwargs records the per-step candidate scores and rewards for inspection.
  • Backends: HF (model-backed per-step logit math is in-process only).
• SASA (API reference, notebook)
  • Description: self-disciplined autoregressive sampling¹¹, fitting a linear subspace in the model's own final-layer space from labeled examples (unpaired classes or paired prompt/response data) and shifting the candidate-token logits by the softmax-normalized margin to that subspace at each step. The candidate set follows a policy (surviving, top_p, or top_k); the attribute is whatever the labels define.
  • Backends: HF (model-backed per-step logit math is in-process only).
• DExperts (API reference, notebook)
  • Description: decoding-time experts¹², re-weighting the base distribution by the log-prob difference between a small expert and anti-expert. Proxy-tuning is the same control with a tuned/untuned small-model pair.
  • Backends: HF (model-backed per-step logit math is in-process only).
• ContrastiveDecoding (API reference, notebook)
  • Description: contrastive decoding¹³, favoring tokens the base (expert) scores higher than a weaker amateur, over an expert-plausibility-masked set.
  • Backends: HF (model-backed per-step logit math is in-process only).
• ConstrainedDecoding (API reference)
  • Description: constrained decoding from one declarative source (JSON schema, regex, EBNF grammar, or a choice set). Every logit the grammar forbids is masked at each step.
  • Backends: HF (client-side xgrammar automaton), vLLM (native structured outputs). A control constructed with an in-memory automaton object is HF-only.
• ValueGuidance (API reference, notebook)
  • Description: the config-first generic over the step shape (candidates → value → normalize → shift). FUDGE, ARGS, RAD, and SASA are assignments of its config.
  • Backends: HF (model-backed per-step logit math is in-process only).
• ContrastiveGuidance (API reference, notebook)
  • Description: the config-first generic over the distribution shape (mix weighted log-prob sources). DExperts, contrastive decoding, and proxy-tuning are assignments of its config.
  • Backends: HF (model-backed per-step logit math is in-process only).
• StoppingRules (API reference, notebook)
  • Description: the config-first generic for stop rules, i.e., substring / token / budget stops as pipeline configuration rather than a class. Since its stops merge into the call's generation parameters, rows halted by them report finish_reason="stop" and the pipeline truncates decoded text at the stop string.
  • Backends: HF, vLLM (stops lower to sampling parameters).

and the following decoding drivers:

• DeAL (API reference, notebook)
  • Description: decoding-time alignment¹⁴, i.e., iterative lookahead beam search with reward-guided beam selection.
  • Backends: HF (beam proposals are in-process only, though the sampled-proposal search runs on vLLM as a SearchDecoding configuration).
• BestOfN (API reference, notebook)
  • Description: best-of-N sampling / re-ranking¹⁵, sampling N full continuations and returning the highest-scoring one under a sequence scorer (pairing with a majority-vote scorer recovers self-consistency).
  • Backends: HF, vLLM.
• BudgetForcing (API reference, notebook)
  • Description: test-time thinking-length control¹⁶, capping each thinking segment, optionally appending extensions ("Wait") to prolong reasoning, then forcing the closing think tag before answering. end_think_token_ids sets the thinking-phase boundary by token id, for a closing-think delimiter that is a special token.
  • Backends: HF, vLLM.
• RoutedDecoding (API reference, notebook)
  • Description: a decoding driver that routes each row to a response plan via a Router over a ProbeSet's readings, and executes the matched plan (canned response, disclaimer prefix, or plain generation). It sits beside PhasedDecoding and SearchDecoding.
  • Backends: HF, offline vLLM (the probe pass needs hidden-state capture, which serve does not return).
• SearchDecoding (API reference, notebook)
  • Description: the config-first generic over the segment shape (propose → score → keep → iterate, with best-of-N defaults). Best-of-N, self-consistency, blockwise controlled decoding, and DeAL are assignments of its config.
  • Backends: HF, vLLM with propose_mode="sample" (beam proposals are HF-only).
• PhasedDecoding (API reference, notebook)
  • Description: the config-first generic over the phase shape (forced / generated segments via a declarative plan grammar). Budget forcing, response prefill, and thinking intervention¹⁷ are assignments of its config. A generate phase ends at its until substring, any token in until_token_ids, or its budget, whichever first.
  • Backends: HF, vLLM.

Some decoding strategies are native to Hugging Face's generate and need no dedicated control. They flow through the default driver via gen_kwargs, for example DoLa decoding (gen_kwargs={"dola_layers": ...}) and watermarking (gen_kwargs={"watermarking_config": ...}).

Generic controls

Alongside the named methods, the output category provides a small family of generic controls, the output analogue of state control's ActivationAdapter. Where a named method (RAD, SASA, DeAL) is a class, a generic exposes the shared component slots through flat, sweepable Args, and a method from the literature becomes an assignment of a config rather than a subclass. Since output controls act either at the step level or by owning the decode loop, and do so over a few distinct shapes of computation, there is one generic per shape:

generic	mechanism	shape	canonical assignments
ValueGuidance	step-level (logits processors)	step	FUDGE, ARGS, RAD-, SASA-equivalents
ContrastiveGuidance	step-level (logits processors)	distribution	DExperts, contrastive decoding, proxy-tuning
SearchDecoding	driver	segment	best-of-N, self-consistency, DeAL-equivalent
PhasedDecoding	driver	phase	budget forcing, response prefill, thinking intervention
StoppingRules	sampling-mapped (stop rules)	none	substring / token / budget stops

The named methods are siblings of these generics rather than children. They are built directly on the same common components, and each keeps the one thing its class adds beyond a config (RAD's cached reward path, SASA's subspace fitting, and so on). When a config earns a name through use, it can be promoted to a small preset subclass over the generic.

Reusable building blocks shared across these methods (candidate policies, per-candidate value functions, full-vocabulary logit sources, sequence scorers, a segment-search driver, a phased driver, composable stopping criteria, and the PrefixKeyedProcessor base) are located in output_control.common.

---

1. Charles Goddard, Shamane Siriwardhana, Malikeh Ehghaghi, Luke Meyers, Vladimir Karpukhin, Brian Benedict, Mark McQuade, and Jacob Solawetz. Arcee's MergeKit: A toolkit for merging large language models. In Franck Dernoncourt, Daniel Preoţiuc-Pietro, and Anastasia Shimorina, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, 477–485. Miami, Florida, US, November 2024. Association for Computational Linguistics. URL: https://aclanthology.org/2024.emnlp-industry.36, doi:10.18653/v1/2024.emnlp-industry.36. ↩

2. Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. TRL: Transformer reinforcement learning. \url https://github.com/huggingface/trl, 2020. ↩

3. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248, 2023. ↩

4. Hieu M. Vu and Tan M. Nguyen. Angular steering: behavior control via rotation in activation space. arXiv preprint arXiv:2510.26243, 2025. ↩

5. Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition. arXiv preprint arXiv:2312.06681, 2023. ↩

6. Bruce W Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. Programming refusal with conditional activation steering. In Proceedings of the 13th International Conference on Learning Representations (ICLR). 2025. ↩

7. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by a single direction. arXiv preprint arXiv:2406.11717, 2024. ↩

8. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. arXiv preprint arXiv:2306.03341, 2023. ↩

9. Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao. Tell your model where to attend: Post-hoc attention steering for LLMs. In The Twelfth International Conference on Learning Representations. 2024. ↩

10. Haikang Deng and Colin Raffel. Reward-augmented decoding: Efficient controlled text generation with a unidirectional reward model. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 11781–11791. Singapore, December 2023. Association for Computational Linguistics. URL: https://aclanthology.org/2023.emnlp-main.721/, doi:10.18653/v1/2023.emnlp-main.721. ↩

11. Ching-Yun Ko, Pin-Yu Chen, Payel Das, Youssef Mroueh, Soham Dan, Georgios Kollias, Subhajit Chaudhury, Tejaswini Pedapati, and Luca Daniel. Large language models can become strong self-detoxifiers. In The Thirteenth International Conference on Learning Representations. 2025. ↩

12. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A Smith, and Yejin Choi. DExperts: Decoding-time controlled text generation with experts and anti-experts. arXiv preprint arXiv:2105.03023, 2021. ↩

13. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. arXiv preprint arXiv:2210.15097, 2022. ↩

14. James Y Huang, Sailik Sengupta, Daniele Bonadiman, Yi-an Lai, Arshit Gupta, Nikolaos Pappas, Saab Mansour, Katrin Kirchhoff, and Dan Roth. DeAL: Decoding-time alignment for large language models. arXiv preprint arXiv:2402.06147, 2024. ↩

15. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. ↩

16. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. S1: Simple test-time scaling. arXiv preprint arXiv:2501.19393, 2025. ↩

17. Tong Wu, Chong Xiang, Jiachen T Wang, G Edward Suh, and Prateek Mittal. Effectively controlling reasoning models through thinking intervention. arXiv preprint arXiv:2503.24370, 2025. ↩