Contrastive Activation Addition (CAA)¶

Paper. Steering Llama 2 via Contrastive Activation Addition

Authors. Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, Alexander Matt Turner

CAA is a state control method that steers model behavior by adding a learned direction vector to the residual stream during generation. The direction is the mean difference between hidden states on paired examples that do and do not exhibit a target behavior. At inference time the vector is added at a single layer with a configurable multiplier, so the sign and magnitude of the multiplier set the direction and degree of steering.

The same mean-difference extraction is the core of recent work on trait steering. Persona Vectors: Monitoring and Controlling Character Traits in Language Models fits directions for character traits by contrasting activations on responses that exhibit the trait against responses that do not, and uses them to monitor and steer chat models. This notebook applies CAA to a persona dimension of that kind, i.e., formality, the axis running from a casual register to a formal one. A single fitted direction serves both ends of the axis, with the sign of multiplier selecting the direction of steering. We fit a formality direction for ibm-granite/granite-4.1-3b from a small pool of contrastive responses, steer the register in both directions on held-out prompts, then save the fitted vector and reuse it with different steering parameters, first in process on the Hugging Face backend and then on the offline vLLM engine.

Method parameters¶

parameter	type	description
steering_vector	SteeringVector	Pre-computed steering vector, used instead of data
data	ContrastivePairs	Paired positive/negative texts used to fit the vector during steer()
train_spec	VectorTrainSpec	Extraction config (method, accumulation mode, rendering, boundary)
layer_id	int	Layer the vector is added at. None selects a layer at roughly 40 percent depth
multiplier	float	Scale on the vector. Positive increases the target behavior, negative decreases it
token_scope	str	Positions the vector is added at: "after_prompt" (default), "all", "last_k", or "from_position"
last_k	int	Number of trailing positions when token_scope="last_k"
from_position	int	Absolute start position when token_scope="from_position"
normalize_vector	bool	L2-normalize the vector before applying
use_norm_preservation	bool	Rescale steered hidden states whose norm increased back to their pre-steering norm

Setup¶

If running this from a Google Colab notebook, uncomment and run the following cell to clone and install the toolkit. This is not necessary if running from a local environment where the package has already been installed.

In [1]:

# !git clone https://github.com/generative-computing/steerability.git
# %cd steerability
# !pip install -q -e .

# !git clone https://github.com/generative-computing/steerability.git # %cd steerability # !pip install -q -e .

The following authentication steps may be necessary to access any gated models (after being granted access by Hugging Face). Uncomment the following if you need to log in to the Hugging Face Hub.

In [2]:

# !pip install -q python-dotenv
# from dotenv import load_dotenv
# import os

# load_dotenv()
# token = os.getenv("HUGGINGFACE_TOKEN")
# from huggingface_hub import login
# login(token=token)

# !pip install -q python-dotenv # from dotenv import load_dotenv # import os # load_dotenv() # token = os.getenv("HUGGINGFACE_TOKEN") # from huggingface_hub import login # login(token=token)

In [3]:

import sys
!{sys.executable} -m pip install -q tabulate

import sys !{sys.executable} -m pip install -q tabulate

In [4]:

import gc
import os
import torch

from transformers import AutoModelForCausalLM, AutoTokenizer

from steerability.algorithms.core.execution import BackendSpec
from steerability.algorithms.core.internals import ContrastivePairs
from steerability.algorithms.core.steering_pipeline import SteeringPipeline
from steerability.algorithms.state_control.common.estimators import MeanDifferenceEstimator
from steerability.algorithms.state_control.common.fit_specs import VectorTrainSpec
from steerability.algorithms.state_control.common.steering_vector import SteeringVector
from steerability.algorithms.state_control.caa.control import CAA

import gc import os import torch from transformers import AutoModelForCausalLM, AutoTokenizer from steerability.algorithms.core.execution import BackendSpec from steerability.algorithms.core.internals import ContrastivePairs from steerability.algorithms.core.steering_pipeline import SteeringPipeline from steerability.algorithms.state_control.common.estimators import MeanDifferenceEstimator from steerability.algorithms.state_control.common.fit_specs import VectorTrainSpec from steerability.algorithms.state_control.common.steering_vector import SteeringVector from steerability.algorithms.state_control.caa.control import CAA

We use ibm-granite/granite-4.1-3b, a compact instruction-tuned model. Fitting runs one forward pass over each side of the contrastive data to read hidden states at every layer, so a GPU with enough memory for the model is recommended.

In [5]:

MODEL_NAME = "ibm-granite/granite-4.1-3b"

MODEL_NAME = "ibm-granite/granite-4.1-3b"

In [6]:

from IPython.display import display, HTML
display(HTML("<style>:root { --jp-notebook-max-width: 100% !important; }</style>"))

from tabulate import tabulate

from IPython.display import display, HTML display(HTML("<style>:root { --jp-notebook-max-width: 100% !important; }</style>")) from tabulate import tabulate

Contrastive data¶

CAA fits its direction from paired texts that differ only in the target behavior. Each pair here shares one user prompt, an everyday request that leaves the register free, and contrasts a formal assistant completion (complete words, measured phrasing, no exclamations) against a casual one (contractions, interjections, colloquial word choice). The two completions of a pair give the same substantive answer at comparable length, so the mean difference isolates the register rather than the content or a length cue, and topics vary across pairs for the same reason.

The formal pool is the positive side of the fit, which sets the sign convention for every control below, i.e., a positive multiplier moves the register towards formal and a negative one towards casual. The pools are hand-written and small for a self-contained demo; the persona vectors pipeline generates much larger pools automatically from a natural-language description of the trait.

In [7]:

formality_pairs = [
    {
        "prompt": "What's a good way to start learning to cook?",
        "formal": "I would suggest beginning with a few versatile techniques, such as roasting vegetables and "
                  "preparing a simple pan sauce. Repeating a small set of reliable dishes builds sound habits, "
                  "after which more ambitious recipes follow quite naturally.",
        "casual": "Honestly, just pick a couple of easy wins like roasted veggies or a simple pan sauce and make "
                  "them a bunch of times. Once you've got a few go-to dishes down, the fancier stuff isn't nearly "
                  "as scary.",
    },
    {
        "prompt": "Any tips for keeping houseplants alive?",
        "formal": "Most houseplants decline from excess water rather than neglect. Allow the soil to dry between "
                  "waterings, provide bright indirect light, and avoid relocating a plant once it has adjusted "
                  "to its position.",
        "casual": "Biggest tip: don't drown them! Let the soil dry out between waterings, stick them somewhere "
                  "bright but out of harsh sun, and once they're settled in a spot, just leave them alone.",
    },
    {
        "prompt": "How should I prepare for a long road trip?",
        "formal": "Have the vehicle serviced beforehand, plan fuel and rest stops at sensible intervals, and "
                  "assemble water, snacks, and a small emergency kit. Downloading offline maps is also advisable, "
                  "as coverage can lapse on rural stretches.",
        "casual": "Get the car checked out first, figure out roughly where you'll stop for gas and breaks, and "
                  "pack water, snacks, and a little emergency kit. Oh, and download offline maps, since signal "
                  "gets spotty out in the sticks.",
    },
    {
        "prompt": "What's a reasonable way to get back into running?",
        "formal": "Begin conservatively, alternating short intervals of running and walking three times per week, "
                  "and increase the total distance by roughly ten percent each week. Consistency at a modest "
                  "volume matters far more than any single ambitious session.",
        "casual": "Start small, like run-walk intervals three times a week, and bump the distance up maybe ten "
                  "percent a week. Showing up regularly beats one big heroic run every time, so don't overdo it "
                  "early!",
    },
    {
        "prompt": "My phone battery drains fast, what can I do?",
        "formal": "Review the battery usage report to identify demanding applications, reduce screen brightness, "
                  "and disable background refresh for applications that do not require it. If the battery is "
                  "several years old, a replacement may be the most effective remedy.",
        "casual": "Check the battery stats to see what's eating it, turn the brightness down, and switch off "
                  "background refresh for apps that don't need it. And if the battery's a few years old, "
                  "honestly, swapping it might be the real fix.",
    },
    {
        "prompt": "How do I make my mornings less chaotic?",
        "formal": "Prepare the evening before by laying out clothing, packing what you will need, and deciding "
                  "on breakfast in advance. A consistent waking time also helps considerably, since much of the "
                  "disorder stems from negotiating decisions while pressed for time.",
        "casual": "Do the prep the night before: lay out your clothes, pack your bag, know what breakfast is "
                  "gonna be. Waking up at the same time every day helps a ton too, 'cause most of the chaos is "
                  "just making decisions while you're rushed.",
    },
    {
        "prompt": "What should I know before adopting a cat?",
        "formal": "Budget for food, litter, and routine veterinary care, and prepare a quiet room where the cat "
                  "can acclimate during the first days. Adult cats often settle more readily than kittens, and "
                  "their temperament is already apparent at adoption.",
        "casual": "Plan on spending for food, litter, and vet visits, and set up a quiet room where the cat can "
                  "chill for the first few days. Grown-up cats often settle in easier than kittens, plus you "
                  "already know what they're like!",
    },
    {
        "prompt": "How can I get better at remembering names?",
        "formal": "Repeat the name immediately upon introduction, use it once or twice in conversation, and "
                  "associate it with a distinctive feature or context. Writing the name down shortly afterwards "
                  "further strengthens the association.",
        "casual": "Say the name back right when you meet them, drop it into the convo once or twice, and tie it "
                  "to something memorable about them. It'll stick way better if you jot it down after, too.",
    },
    {
        "prompt": "What's a good approach to decluttering?",
        "formal": "Proceed one category at a time rather than one room at a time, retain the items you use or "
                  "genuinely value, and remove discarded items from the house promptly. Short, regular sessions "
                  "tend to be more sustainable than a single exhausting purge.",
        "casual": "Go one category at a time instead of room by room, keep the stuff you actually use or love, "
                  "and don't let the discard pile hang around, get it out of the house fast. A bunch of short "
                  "sessions beats one giant exhausting purge, trust me.",
    },
    {
        "prompt": "How do I brew better coffee at home?",
        "formal": "Purchase whole beans, grind them immediately before brewing, and weigh both the coffee and "
                  "the water; a ratio near one to sixteen is a dependable starting point. Water just off the "
                  "boil extracts more evenly than water at a full boil.",
        "casual": "Get whole beans and grind them right before you brew, and weigh your coffee and water, "
                  "something like one to sixteen is a solid start. And don't pour the water right at a boil; "
                  "let it sit a sec first, it makes a real difference.",
    },
    {
        "prompt": "Any tips for writing a short bio about myself?",
        "formal": "Write in the third person, lead with your current role, add one or two notable "
                  "accomplishments, and close with a brief personal detail. Aim for three or four sentences and "
                  "set the draft aside before a final revision.",
        "casual": "Write it in the third person, kick off with what you do now, toss in an accomplishment or "
                  "two, and end with one fun personal bit. Keep it to three or four sentences, and don't do "
                  "the final pass until you've slept on it.",
    },
    {
        "prompt": "How do I keep bananas from ripening too fast?",
        "formal": "Separate the bananas from the bunch, keep them away from other fruit, and wrap each stem in "
                  "plastic to slow the release of ethylene. Once they reach the desired ripeness, refrigeration "
                  "halts the process, although the peel will darken.",
        "casual": "Split them off the bunch, keep them away from other fruit, and wrap the stems in a bit of "
                  "plastic; that slows down the ethylene. Once they're ripe enough, chuck them in the fridge. "
                  "The peel goes dark but the inside's fine.",
    },
    {
        "prompt": "What's a sensible way to back up my photos?",
        "formal": "Follow the three-two-one principle: three copies of the data, on two different types of "
                  "storage, with one copy kept off site. In practice, an automatic cloud backup combined with a "
                  "periodic copy to an external drive satisfies this comfortably.",
        "casual": "Go with the three-two-one thing: three copies, two kinds of storage, one of them off site. "
                  "Basically, let a cloud service back stuff up automatically and copy everything to an external "
                  "drive every so often, and you're covered.",
    },
    {
        "prompt": "How can I make small talk less awkward?",
        "formal": "Ask open questions about the immediate context, listen for details worth pursuing, and offer "
                  "small observations of your own so the exchange does not resemble an interview. Brief silences "
                  "are normal and rarely as noticeable as they feel.",
        "casual": "Ask open questions about whatever's going on around you, actually listen for threads to pull "
                  "on, and share little things yourself so it doesn't turn into an interview. And don't sweat "
                  "the pauses; nobody notices them like you do.",
    },
    {
        "prompt": "What stretches help after sitting all day?",
        "formal": "Prioritize the areas that shorten while seated: the hip flexors, the chest, and the "
                  "hamstrings. A kneeling hip flexor stretch, a doorway chest stretch, and a standing hamstring "
                  "stretch, each held for thirty seconds or so, cover these well.",
        "casual": "Hit the spots that get tight from sitting: hips, chest, hamstrings. A kneeling hip flexor "
                  "stretch, a doorway chest stretch, and a standing hamstring stretch, maybe thirty seconds "
                  "each, and you're pretty much sorted.",
    },
    {
        "prompt": "How do I pick a good watermelon?",
        "formal": "Choose a melon that feels heavy for its size, with a large creamy yellow patch where it "
                  "rested on the ground, and a dull rather than glossy rind. A deep hollow sound when tapped is "
                  "a further favorable indication.",
        "casual": "Grab one that feels heavy for its size, look for a big creamy yellow spot where it sat on "
                  "the ground, and skip the shiny ones, they're usually underripe. If it sounds nice and "
                  "hollow when you knock on it, even better!",
    },
    {
        "prompt": "Any advice for hosting dinner for the first time?",
        "formal": "Choose a dish you have prepared successfully before, complete whatever can be done in "
                  "advance, and set the table early. Guests take their cue from the host, so a composed welcome "
                  "matters more than an elaborate menu.",
        "casual": "Make something you've cooked before and know turns out fine, prep whatever you can ahead of "
                  "time, and set the table early. People mostly vibe off the host, so if you're relaxed, nobody "
                  "cares how fancy the food is.",
    },
    {
        "prompt": "How do I stop hitting snooze in the morning?",
        "formal": "Place the alarm across the room so that rising is required to silence it, keep a consistent "
                  "sleep schedule, and seek bright light promptly upon waking. An earlier bedtime addresses the "
                  "underlying cause more directly than any alarm strategy.",
        "casual": "Stick your alarm across the room so you have to get up to shut it off, keep your sleep "
                  "schedule steady, and get some bright light going as soon as you're up. And honestly, going "
                  "to bed earlier fixes the actual problem.",
    },
]

train_pairs = ContrastivePairs(
    prompts=[pair["prompt"] for pair in formality_pairs],
    positives=[pair["formal"] for pair in formality_pairs],
    negatives=[pair["casual"] for pair in formality_pairs],
)

print(f"{len(train_pairs.positives)} contrastive pairs")

formality_pairs = [ { "prompt": "What's a good way to start learning to cook?", "formal": "I would suggest beginning with a few versatile techniques, such as roasting vegetables and " "preparing a simple pan sauce. Repeating a small set of reliable dishes builds sound habits, " "after which more ambitious recipes follow quite naturally.", "casual": "Honestly, just pick a couple of easy wins like roasted veggies or a simple pan sauce and make " "them a bunch of times. Once you've got a few go-to dishes down, the fancier stuff isn't nearly " "as scary.", }, { "prompt": "Any tips for keeping houseplants alive?", "formal": "Most houseplants decline from excess water rather than neglect. Allow the soil to dry between " "waterings, provide bright indirect light, and avoid relocating a plant once it has adjusted " "to its position.", "casual": "Biggest tip: don't drown them! Let the soil dry out between waterings, stick them somewhere " "bright but out of harsh sun, and once they're settled in a spot, just leave them alone.", }, { "prompt": "How should I prepare for a long road trip?", "formal": "Have the vehicle serviced beforehand, plan fuel and rest stops at sensible intervals, and " "assemble water, snacks, and a small emergency kit. Downloading offline maps is also advisable, " "as coverage can lapse on rural stretches.", "casual": "Get the car checked out first, figure out roughly where you'll stop for gas and breaks, and " "pack water, snacks, and a little emergency kit. Oh, and download offline maps, since signal " "gets spotty out in the sticks.", }, { "prompt": "What's a reasonable way to get back into running?", "formal": "Begin conservatively, alternating short intervals of running and walking three times per week, " "and increase the total distance by roughly ten percent each week. Consistency at a modest " "volume matters far more than any single ambitious session.", "casual": "Start small, like run-walk intervals three times a week, and bump the distance up maybe ten " "percent a week. Showing up regularly beats one big heroic run every time, so don't overdo it " "early!", }, { "prompt": "My phone battery drains fast, what can I do?", "formal": "Review the battery usage report to identify demanding applications, reduce screen brightness, " "and disable background refresh for applications that do not require it. If the battery is " "several years old, a replacement may be the most effective remedy.", "casual": "Check the battery stats to see what's eating it, turn the brightness down, and switch off " "background refresh for apps that don't need it. And if the battery's a few years old, " "honestly, swapping it might be the real fix.", }, { "prompt": "How do I make my mornings less chaotic?", "formal": "Prepare the evening before by laying out clothing, packing what you will need, and deciding " "on breakfast in advance. A consistent waking time also helps considerably, since much of the " "disorder stems from negotiating decisions while pressed for time.", "casual": "Do the prep the night before: lay out your clothes, pack your bag, know what breakfast is " "gonna be. Waking up at the same time every day helps a ton too, 'cause most of the chaos is " "just making decisions while you're rushed.", }, { "prompt": "What should I know before adopting a cat?", "formal": "Budget for food, litter, and routine veterinary care, and prepare a quiet room where the cat " "can acclimate during the first days. Adult cats often settle more readily than kittens, and " "their temperament is already apparent at adoption.", "casual": "Plan on spending for food, litter, and vet visits, and set up a quiet room where the cat can " "chill for the first few days. Grown-up cats often settle in easier than kittens, plus you " "already know what they're like!", }, { "prompt": "How can I get better at remembering names?", "formal": "Repeat the name immediately upon introduction, use it once or twice in conversation, and " "associate it with a distinctive feature or context. Writing the name down shortly afterwards " "further strengthens the association.", "casual": "Say the name back right when you meet them, drop it into the convo once or twice, and tie it " "to something memorable about them. It'll stick way better if you jot it down after, too.", }, { "prompt": "What's a good approach to decluttering?", "formal": "Proceed one category at a time rather than one room at a time, retain the items you use or " "genuinely value, and remove discarded items from the house promptly. Short, regular sessions " "tend to be more sustainable than a single exhausting purge.", "casual": "Go one category at a time instead of room by room, keep the stuff you actually use or love, " "and don't let the discard pile hang around, get it out of the house fast. A bunch of short " "sessions beats one giant exhausting purge, trust me.", }, { "prompt": "How do I brew better coffee at home?", "formal": "Purchase whole beans, grind them immediately before brewing, and weigh both the coffee and " "the water; a ratio near one to sixteen is a dependable starting point. Water just off the " "boil extracts more evenly than water at a full boil.", "casual": "Get whole beans and grind them right before you brew, and weigh your coffee and water, " "something like one to sixteen is a solid start. And don't pour the water right at a boil; " "let it sit a sec first, it makes a real difference.", }, { "prompt": "Any tips for writing a short bio about myself?", "formal": "Write in the third person, lead with your current role, add one or two notable " "accomplishments, and close with a brief personal detail. Aim for three or four sentences and " "set the draft aside before a final revision.", "casual": "Write it in the third person, kick off with what you do now, toss in an accomplishment or " "two, and end with one fun personal bit. Keep it to three or four sentences, and don't do " "the final pass until you've slept on it.", }, { "prompt": "How do I keep bananas from ripening too fast?", "formal": "Separate the bananas from the bunch, keep them away from other fruit, and wrap each stem in " "plastic to slow the release of ethylene. Once they reach the desired ripeness, refrigeration " "halts the process, although the peel will darken.", "casual": "Split them off the bunch, keep them away from other fruit, and wrap the stems in a bit of " "plastic; that slows down the ethylene. Once they're ripe enough, chuck them in the fridge. " "The peel goes dark but the inside's fine.", }, { "prompt": "What's a sensible way to back up my photos?", "formal": "Follow the three-two-one principle: three copies of the data, on two different types of " "storage, with one copy kept off site. In practice, an automatic cloud backup combined with a " "periodic copy to an external drive satisfies this comfortably.", "casual": "Go with the three-two-one thing: three copies, two kinds of storage, one of them off site. " "Basically, let a cloud service back stuff up automatically and copy everything to an external " "drive every so often, and you're covered.", }, { "prompt": "How can I make small talk less awkward?", "formal": "Ask open questions about the immediate context, listen for details worth pursuing, and offer " "small observations of your own so the exchange does not resemble an interview. Brief silences " "are normal and rarely as noticeable as they feel.", "casual": "Ask open questions about whatever's going on around you, actually listen for threads to pull " "on, and share little things yourself so it doesn't turn into an interview. And don't sweat " "the pauses; nobody notices them like you do.", }, { "prompt": "What stretches help after sitting all day?", "formal": "Prioritize the areas that shorten while seated: the hip flexors, the chest, and the " "hamstrings. A kneeling hip flexor stretch, a doorway chest stretch, and a standing hamstring " "stretch, each held for thirty seconds or so, cover these well.", "casual": "Hit the spots that get tight from sitting: hips, chest, hamstrings. A kneeling hip flexor " "stretch, a doorway chest stretch, and a standing hamstring stretch, maybe thirty seconds " "each, and you're pretty much sorted.", }, { "prompt": "How do I pick a good watermelon?", "formal": "Choose a melon that feels heavy for its size, with a large creamy yellow patch where it " "rested on the ground, and a dull rather than glossy rind. A deep hollow sound when tapped is " "a further favorable indication.", "casual": "Grab one that feels heavy for its size, look for a big creamy yellow spot where it sat on " "the ground, and skip the shiny ones, they're usually underripe. If it sounds nice and " "hollow when you knock on it, even better!", }, { "prompt": "Any advice for hosting dinner for the first time?", "formal": "Choose a dish you have prepared successfully before, complete whatever can be done in " "advance, and set the table early. Guests take their cue from the host, so a composed welcome " "matters more than an elaborate menu.", "casual": "Make something you've cooked before and know turns out fine, prep whatever you can ahead of " "time, and set the table early. People mostly vibe off the host, so if you're relaxed, nobody " "cares how fancy the food is.", }, { "prompt": "How do I stop hitting snooze in the morning?", "formal": "Place the alarm across the room so that rising is required to silence it, keep a consistent " "sleep schedule, and seek bright light promptly upon waking. An earlier bedtime addresses the " "underlying cause more directly than any alarm strategy.", "casual": "Stick your alarm across the room so you have to get up to shut it off, keep your sleep " "schedule steady, and get some bright light going as soon as you're up. And honestly, going " "to bed earlier fixes the actual problem.", }, ] train_pairs = ContrastivePairs( prompts=[pair["prompt"] for pair in formality_pairs], positives=[pair["formal"] for pair in formality_pairs], negatives=[pair["casual"] for pair in formality_pairs], ) print(f"{len(train_pairs.positives)} contrastive pairs")

18 contrastive pairs

We hold out a handful of prompts for evaluation. None of them appear in the fitting data, and each is an ordinary request that a chat model can answer in either register, so the register of the response is free to move under steering.

In [8]:

eval_prompts = [
    "Why do we get songs stuck in our heads?",
    "Give me some tips for my first job interview next week.",
    "What should I cook for a quick weeknight dinner?",
    "My laptop has gotten slow, what can I do about it?",
    "How does sourdough bread rise without commercial yeast?",
    "How do I get better at waking up early?",
]

eval_prompts = [ "Why do we get songs stuck in our heads?", "Give me some tips for my first job interview next week.", "What should I cook for a quick weeknight dinner?", "My laptop has gotten slow, what can I do about it?", "How does sourdough bread rise without commercial yeast?", "How do I get better at waking up early?", ]

Baseline behavior¶

We load the model and tokenizer once and share them across every pipeline in this notebook by passing the loaded objects to each SteeringPipeline as model= and tokenizer=, so the pipelines differ only in their controls. Generation runs through pipeline.generate(messages=...) everywhere, which applies the chat template, generates, and returns only the completion; the baseline uses a control-free pipeline so that a single code path serves the whole notebook.

In [9]:

model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto", dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
device = model.device

gen_params = {
    "max_new_tokens": 120,
    "do_sample": False,
    "repetition_penalty": 1.1,
    "pad_token_id": tokenizer.eos_token_id,
}

model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto", dtype=torch.bfloat16) tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) device = model.device gen_params = { "max_new_tokens": 120, "do_sample": False, "repetition_penalty": 1.1, "pad_token_id": tokenizer.eos_token_id, }

In [10]:

baseline_pipeline = SteeringPipeline(model=model, tokenizer=tokenizer)
baseline_pipeline.steer()

baseline_responses = baseline_pipeline.generate(
    messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts],
    **gen_params,
)

print(tabulate(
    [[prompt, response] for prompt, response in zip(eval_prompts, baseline_responses)],
    headers=["prompt", "baseline response"],
    tablefmt="grid",
    maxcolwidths=[28, 84],
))

baseline_pipeline = SteeringPipeline(model=model, tokenizer=tokenizer) baseline_pipeline.steer() baseline_responses = baseline_pipeline.generate( messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts], **gen_params, ) print(tabulate( [[prompt, response] for prompt, response in zip(eval_prompts, baseline_responses)], headers=["prompt", "baseline response"], tablefmt="grid", maxcolwidths=[28, 84], ))

+------------------------------+--------------------------------------------------------------------------------------+
| prompt                       | baseline response                                                                    |
+==============================+======================================================================================+
| Why do we get songs stuck in | Songs getting stuck in your head, commonly referred to as “earworms,” occur due to a |
| our heads?                   | variety of psychological and neurological factors:                                   |
|                              | 1. **Repetition and Memorability**: Songs that are catchy, repetitive, or have a     |
|                              | strong emotional impact are more likely to be remembered. The repetition helps       |
|                              | encode the music into long-term memory.                                              |
|                              | 2. **Emotional Resonance**: Music that evokes strong emotions—whether positive or    |
|                              | negative—can leave a lasting impression on the brain. Emotional connections can make |
|                              | certain tunes more memorable and harder to shake off.                                |
|                              | 3. **Neural Patterns**: When you                                                     |
+------------------------------+--------------------------------------------------------------------------------------+
| Give me some tips for my     | Certainly! Preparing thoroughly for your first job interview can significantly boost |
| first job interview next     | your confidence and increase your chances of success. Here are some essential tips:  |
| week.                        | ### 1. **Research the Company**                                                      |
|                              |    - **Understand Their Mission:** Know what the company stands for, its values,     |
|                              | products/services, and recent news or achievements.                                  |
|                              |    - **Learn About the Role:** Understand the responsibilities of the position       |
|                              | you’re applying for. Match your skills and experiences to these requirements.        |
|                              | ### 2. **Know Your Resume Inside Out**                                               |
|                              |    - Be prepared to discuss any aspect of your resume. Practice explaining how each  |
|                              | experience                                                                           |
+------------------------------+--------------------------------------------------------------------------------------+
| What should I cook for a     | For a quick weeknight dinner, consider one of these easy and satisfying options:     |
| quick weeknight dinner?      | 1. **Stir-Fry**: Use your favorite vegetables (like bell peppers, broccoli, carrots, |
|                              | and snap peas) along with protein such as chicken, shrimp, tofu, or beef. Stir-fry   |
|                              | in a pan with a little oil, garlic, and soy sauce or teriyaki sauce. Serve over rice |
|                              | or noodles.                                                                          |
|                              | 2. **Pasta Primavera**: Cook pasta according to package instructions. Sauté fresh or |
|                              | frozen mixed vegetables with olive oil, garlic, and herbs like basil or parsley. T   |
+------------------------------+--------------------------------------------------------------------------------------+
| My laptop has gotten slow,   | There are several steps you can take to speed up a slow laptop:                      |
| what can I do about it?      | 1. **Remove Unnecessary Programs**: Uninstall programs that you no longer use. These |
|                              | can run in the background and consume resources.                                     |
|                              | 2. **Disable Startup Programs**: Many programs automatically start when your         |
|                              | computer boots up. Limiting these can speed up the login process.                    |
|                              |    - On Windows: Use Task Manager (Ctrl+Shift+Esc), go to the "Startup" tab, and     |
|                              | disable unnecessary items.                                                           |
|                              |    - On macOS: Go to System Preferences > Users & Groups > Login Items and remove    |
|                              | unwanted applications.                                                               |
|                              | 3. **                                                                                |
+------------------------------+--------------------------------------------------------------------------------------+
| How does sourdough bread     | Sourdough bread rises through a natural fermentation process that involves wild      |
| rise without commercial      | yeasts and lactic acid bacteria present in the flour and environment. Here’s how it  |
| yeast?                       | works:                                                                               |
|                              | 1. **Wild Yeast and Lactic Acid Bacteria**: When you start making sourdough, you     |
|                              | create a “starter” by mixing flour with water. Over time, naturally occurring wild   |
|                              | yeasts (primarily Saccharomyces cerevisiae) and lactic acid bacteria (such as        |
|                              | Lactobacillus species) begin to colonize this mixture.                               |
|                              | 2. **Fermentation Process**: These microorganisms feed on                            |
+------------------------------+--------------------------------------------------------------------------------------+
| How do I get better at       | Improving your ability to wake up early requires a combination of lifestyle          |
| waking up early?             | adjustments, habit formation, and sometimes, technological aids. Here are several    |
|                              | strategies that can help you become more consistent with an early wake-up time:      |
|                              | ### 1. **Establish a Consistent Sleep Schedule**                                     |
|                              |    - Go to bed and wake up at the same time every day, even on weekends. This helps  |
|                              | regulate your body's internal clock (circadian rhythm).                              |
|                              | ### 2. **Create a Bedtime Routine**                                                  |
|                              |    - Develop a relaxing pre-sleep routine to signal to your body that it’s time to   |
|                              | wind down. This                                                                      |
+------------------------------+--------------------------------------------------------------------------------------+

Fitting the formality direction¶

MeanDifferenceEstimator renders each pair through the model's chat template (prompt_format="chat_completion" renders the prompt as a user turn and appends the completion after the generation prompt), runs one forward pass over each side, and takes the mean difference between the formal and casual activations at every layer. The result is a SteeringVector holding one direction per layer.

The accumulate="last_token" argument reads each example's hidden state at the final completion token. This is the extraction point of the original CAA setup, where each completion is a one-token multiple-choice answer; with full responses the final token comes after the model has processed the entire completion, so its state summarizes the response's register.

Note that passing data= and train_spec= to CAA runs this same fit inside steer(). We fit the vector standalone here so that one fit serves every steering configuration below and can be saved for the engine section.

In [11]:

train_spec = VectorTrainSpec(
    method="mean_diff",
    accumulate="last_token",
    prompt_format="chat_completion",
)

formality_vector = MeanDifferenceEstimator().fit(
    model,
    tokenizer,
    data=train_pairs,
    spec=train_spec,
)

print(f"Fitted a formality direction for {len(formality_vector.directions)} layers")

train_spec = VectorTrainSpec( method="mean_diff", accumulate="last_token", prompt_format="chat_completion", ) formality_vector = MeanDifferenceEstimator().fit( model, tokenizer, data=train_pairs, spec=train_spec, ) print(f"Fitted a formality direction for {len(formality_vector.directions)} layers")

Fitted a formality direction for 40 layers

Steering towards a formal register¶

CAA adds multiplier * v to the residual stream at the output of one layer. The formal pool is the positive side of the fit, so a positive multiplier moves the register towards formal. We add at a mid-depth layer; if layer_id is omitted the control selects a layer at roughly 40 percent depth, the depth range the CAA paper found most effective. The default token_scope="after_prompt" applies the vector at generated positions only, leaving the prompt's own representation untouched. use_norm_preservation=True rescales steered hidden states whose norm increased back to their pre-steering norm.

Note that useful values of multiplier depend on the model, the layer, and the accumulation mode, so the value below is a starting point worth sweeping. Values that are too small change little, and values that are too large degrade fluency, which surfaces as repetition, garbled phrasing, or drift away from the question.

In [12]:

LAYER_ID = 16
MULTIPLIER = 4.0

caa_formal = CAA(
    steering_vector=formality_vector,
    layer_id=LAYER_ID,
    multiplier=MULTIPLIER,
    use_norm_preservation=True,
)

formal_pipeline = SteeringPipeline(controls=[caa_formal], model=model, tokenizer=tokenizer)
formal_pipeline.steer()

formal_responses = formal_pipeline.generate(
    messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts],
    **gen_params,
)

LAYER_ID = 16 MULTIPLIER = 4.0 caa_formal = CAA( steering_vector=formality_vector, layer_id=LAYER_ID, multiplier=MULTIPLIER, use_norm_preservation=True, ) formal_pipeline = SteeringPipeline(controls=[caa_formal], model=model, tokenizer=tokenizer) formal_pipeline.steer() formal_responses = formal_pipeline.generate( messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts], **gen_params, )

Steering towards a casual register¶

Flipping the sign subtracts the direction, moving the register towards the casual end of the same axis. One fitted vector therefore serves both ends of the dimension, and the two steered configurations differ only in the sign of multiplier. Movement in both directions along a single vector also serves as the causal check that the direction captures the register rather than an artifact of either pool.

In [13]:

caa_casual = CAA(
    steering_vector=formality_vector,
    layer_id=LAYER_ID,
    multiplier=-MULTIPLIER,
    use_norm_preservation=True,
)

casual_pipeline = SteeringPipeline(controls=[caa_casual], model=model, tokenizer=tokenizer)
casual_pipeline.steer()

casual_responses = casual_pipeline.generate(
    messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts],
    **gen_params,
)

caa_casual = CAA( steering_vector=formality_vector, layer_id=LAYER_ID, multiplier=-MULTIPLIER, use_norm_preservation=True, ) casual_pipeline = SteeringPipeline(controls=[caa_casual], model=model, tokenizer=tokenizer) casual_pipeline.steer() casual_responses = casual_pipeline.generate( messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts], **gen_params, )

Comparing responses¶

The table places the three configurations side by side on each held-out prompt. The relevant reading is the phrasing rather than the substance of the advice, and since decoding is greedy, differences between the columns come from the steering alone.

In [14]:

table_rows = []
for i, prompt in enumerate(eval_prompts):
    table_rows.append([prompt, baseline_responses[i], casual_responses[i], formal_responses[i]])

print(tabulate(
    table_rows,
    headers=["prompt", "baseline", f"multiplier={-MULTIPLIER}", f"multiplier={MULTIPLIER}"],
    tablefmt="grid",
    maxcolwidths=[28, 42, 42, 42],
))

table_rows = [] for i, prompt in enumerate(eval_prompts): table_rows.append([prompt, baseline_responses[i], casual_responses[i], formal_responses[i]]) print(tabulate( table_rows, headers=["prompt", "baseline", f"multiplier={-MULTIPLIER}", f"multiplier={MULTIPLIER}"], tablefmt="grid", maxcolwidths=[28, 42, 42, 42], ))

+------------------------------+--------------------------------------------+--------------------------------------------+--------------------------------------------+
| prompt                       | baseline                                   | multiplier=-4.0                            | multiplier=4.0                             |
+==============================+============================================+============================================+============================================+
| Why do we get songs stuck in | Songs getting stuck in your head, commonly | Songs getting stuck in your head is a      | Songs getting stuck in the mind, commonly  |
| our heads?                   | referred to as “earworms,” occur due to a  | common experience, and there's actually    | referred to as “earworms,” are thought to  |
|                              | variety of psychological and neurological  | some science behind it. This phenomenon    | result from the repetition of musical      |
|                              | factors:                                   | has a name too—it's called an "earworm."   | elements that evoke vivid mental imagery   |
|                              | 1. **Repetition and Memorability**: Songs  | And no, your brain isn't trying to mess    | and associated memories. Repetition of     |
|                              | that are catchy, repetitive, or have a     | with you. There are a few reasons why that | musical stimuli can activate auditory      |
|                              | strong emotional impact are more likely to | catchy tune won't leave:                   | cortical areas, leading to the generation  |
|                              | be remembered. The repetition helps encode | 1. **It's Catchy**: Sometimes, the song    | of internal representations of heard       |
|                              | the music into long-term memory.           | just hits a good spot on the musical scale | sounds (Melzack & Wall, 1969). These       |
|                              | 2. **Emotional Resonance**: Music that     | that's easy to hum or sing along to. And   | internally generated auditory images may   |
|                              | evokes strong emotions—whether positive or | once it starts playing, it's like your     | persist even when the original stimulus is |
|                              | negative—can leave a lasting impression on | brain's version of a personal playlist.    | no longer present (Oliver et al., 2002).   |
|                              | the brain. Emotional connections can make  | 2. **Your Brain's Favorite**: Maybe you    | The persistence of these mental images may |
|                              | certain tunes more memorable and harder to | heard                                      | be accompanied by subjective feelings of   |
|                              | shake off.                                 |                                            | familiarity, emotion, and autobiographical |
|                              | 3. **Neural Patterns**: When you           |                                            | memory retrieval (Freud,                   |
+------------------------------+--------------------------------------------+--------------------------------------------+--------------------------------------------+
| Give me some tips for my     | Certainly! Preparing thoroughly for your   | Certainly! Preparing for your first job    | Certainly! Preparing for a job interview   |
| first job interview next     | first job interview can significantly      | interview can be nerve-wracking, but       | is essential to ensure that you present    |
| week.                        | boost your confidence and increase your    | you've got this. Here are some tips to     | yourself effectively and maximize your     |
|                              | chances of success. Here are some          | help you shine:                            | chances of success. Here are some general  |
|                              | essential tips:                            | ### 1. **Research the Company**            | pieces of advice:                          |
|                              | ### 1. **Research the Company**            |    - Make sure you know what the company   | 1. **Research the Organization**           |
|                              |    - **Understand Their Mission:** Know    | does, their mission, and maybe a few of    |    - Obtain information regarding the      |
|                              | what the company stands for, its values,   | their biggest projects lately. Show them   | organization, including its history,       |
|                              | products/services, and recent news or      | you're interested in being part of the     | mission statement, products, services,     |
|                              | achievements.                              | team.                                      | customers, the workforce, the geographical |
|                              |    - **Learn About the Role:** Understand  | ### 2. **Know Your Resume**                | presence, the system of internal control,  |
|                              | the responsibilities of the position       |    - Practice talking about your           | the organizational structure, the market   |
|                              | you’re applying for. Match your skills and | experience. Be ready to explain how each   | position, financial condition, and the     |
|                              | experiences to these requirements.         | job (or school project, volunteer work)    | audit and accounting policies.             |
|                              | ### 2. **Know Your Resume Inside Out**     | makes you perfect for this role.           |    Thoroughly review the information on    |
|                              |    - Be prepared to discuss any aspect of  | ### 3.                                     | the Internet, company publications and     |
|                              | your resume. Practice explaining how each  |                                            | news reports.                              |
|                              | experience                                 |                                            | 2. **Simulate an Interview**               |
|                              |                                            |                                            |    Prepare responses to                    |
+------------------------------+--------------------------------------------+--------------------------------------------+--------------------------------------------+
| What should I cook for a     | For a quick weeknight dinner, consider one | For a quick weeknight dinner, you have     | For a quick weeknight dinner, consider     |
| quick weeknight dinner?      | of these easy and satisfying options:      | lots of options depending on what you've   | preparing one of the following recipes:    |
|                              | 1. **Stir-Fry**: Use your favorite         | got in the fridge and what's easy to find  | 1. **Grilled Chicken with Thyme and        |
|                              | vegetables (like bell peppers, broccoli,   | at your local store. Here are a few ideas  | Olives**                                   |
|                              | carrots, and snap peas) along with protein | that don't take too long:                  |    - Marinate chicken breasts in a mixture |
|                              | such as chicken, shrimp, tofu, or beef.    | ### 1. Pasta Night                         | of olive oil, ground black pepper, salt,   |
|                              | Stir-fry in a pan with a little oil,       | - **Spaghetti with marinara & cheese** -   | garlic, and dried thyme. Grill the         |
|                              | garlic, and soy sauce or teriyaki sauce.   | Just boil some pasta, make or buy marinara | marinated pieces until cooked through.     |
|                              | Serve over rice or noodles.                | sauce, melt some mozzarella (or provolone  | 2. **Stuffed Peppers**                     |
|                              | 2. **Pasta Primavera**: Cook pasta         | if you like), toss it together. Bam,       |    - Prepare a filling consisting of       |
|                              | according to package instructions. Sauté   | dinner.                                    | minced meat, onion, tomato purée, rice,    |
|                              | fresh or frozen mixed vegetables with      | ### 2. Stir-Fry                            | spices, and seasoning. Steam or bake the   |
|                              | olive oil, garlic, and herbs like basil or | - **Chicken & Veggies** - Grab some        | prepared mixture in individual containers  |
|                              | parsley. T                                 | chicken breast, rice noodles, broccoli     | filled with the vegetable stuffing of      |
|                              |                                            |                                            | whole canned peeled tomatoes.              |
|                              |                                            |                                            | 3. **Ste                                   |
+------------------------------+--------------------------------------------+--------------------------------------------+--------------------------------------------+
| My laptop has gotten slow,   | There are several steps you can take to    | There are several things you can try to    | There are several steps you can take to    |
| what can I do about it?      | speed up a slow laptop:                    | speed up your laptop. Here’s a list of     | improve the performance of your laptop:    |
|                              | 1. **Remove Unnecessary Programs**:        | common solutions:                          | 1. **Restart Your Laptop:** Restarting     |
|                              | Uninstall programs that you no longer use. | ### 1. **Close the Apps You Don't Need**   | your laptop is an effective and            |
|                              | These can run in the background and        |    - Open Task Manager (Windows) or        | straightforward method to free memory      |
|                              | consume resources.                         | Activity Monitor (Mac), and close any      | occupied by inactive applications. It      |
|                              | 2. **Disable Startup Programs**: Many      | browser tabs, games, or apps you ain't     | closes all application programs and        |
|                              | programs automatically start when your     | using. Those resource hogs just sit there  | releases memory that is no longer in use.  |
|                              | computer boots up. Limiting these can      | eating RAM.                                | 2. **Reduce the Use of Desktop             |
|                              | speed up the login process.                | ### 2. **Delete Junk Files**               | Environment:**                             |
|                              |    - On Windows: Use Task Manager          |    - Go through your Downloads folder,     |    - Minimize the icons on the desktop and |
|                              | (Ctrl+Shift+Esc), go to the "Startup" tab, | Pictures, Music – anywhere that's not      | the use of dialog boxes by reducing the    |
|                              | and disable unnecessary items.             | Documents or Desktop. Delete stuff you     | amount of information displayed on the     |
|                              |    - On macOS: Go to System Preferences >  | don't need. On Windows, hit `Win + R`,     | screen.                                    |
|                              | Users & Groups > Login Items and remove    | type `                                     | 3. **Regularly Remove Temporary Files:**   |
|                              | unwanted applications.                     |                                            |    - Dynamic allocation of storage is      |
|                              | 3. **                                      |                                            | performed automatically by the operating   |
|                              |                                            |                                            | system. Temporary files generated by       |
+------------------------------+--------------------------------------------+--------------------------------------------+--------------------------------------------+
| How does sourdough bread     | Sourdough bread rises through a natural    | Sourdough bread rises without commercial   | Sourdough bread rises without the use of   |
| rise without commercial      | fermentation process that involves wild    | yeast by using a natural leavening process | commercial yeast through a natural         |
| yeast?                       | yeasts and lactic acid bacteria present in | with wild yeast and some good old lactic   | fermentation process involving wild        |
|                              | the flour and environment. Here’s how it   | acid bacteria. It's all about the magic of | microorganisms. The key steps and          |
|                              | works:                                     | fermentation. Here’s how it works:         | mechanisms involved are:                   |
|                              | 1. **Wild Yeast and Lactic Acid            | ### 1. The Starter (Big Ass Starter)       | 1. **Formation of the Sourdough Starter:** |
|                              | Bacteria**: When you start making          | First off, you need something called a     |    A sourdough starter is prepared by      |
|                              | sourdough, you create a “starter” by       | "starter." It's not actual live yeast like | mixing flour with water, allowing the      |
|                              | mixing flour with water. Over time,        | in your bread flour, but more like a       | spontaneous growth of lactic acid bacteria |
|                              | naturally occurring wild yeasts (primarily | little colony of wild microorganisms       | (LAB) and ethanol-producing yees (mainly   |
|                              | Saccharomyces cerevisiae) and lactic acid  | hanging out in a mix of flour and water.   | *Saccharomyces* species). After several    |
|                              | bacteria (such as Lactobacillus species)   | You let that sit for a while, maybe a week | days of maturation, the mixture exhibits   |
|                              | begin to colonize this mixture.            | or two, and you start feeding it every now | an acidic pH and develops effervent        |
|                              | 2. **Fermentation Process**: These         | and then                                   | properties due to the production of carbon |
|                              | microorganisms feed on                     |                                            | dioxide.                                   |
|                              |                                            |                                            | 2. **Levain Preparation:**                 |
+------------------------------+--------------------------------------------+--------------------------------------------+--------------------------------------------+
| How do I get better at       | Improving your ability to wake up early    | Improving your morning routine and getting | Improving your ability to wake up early    |
| waking up early?             | requires a combination of lifestyle        | up earlier can be a game-changer. Here are | requires a combination of consistent       |
|                              | adjustments, habit formation, and          | some tips to help you wake up on the right | habits, environmental adjustments, and the |
|                              | sometimes, technological aids. Here are    | side of the bed:                           | development of positive mental attitudes.  |
|                              | several strategies that can help you       | ### 1. **Set a Consistent Schedule**       | Consider the following strategies:         |
|                              | become more consistent with an early wake- |    - Try to go to bed at roughly the same  | ### 1. **Establish a Regular Sleep         |
|                              | up time:                                   | time every night, even on weekends. And    | Schedule**                                 |
|                              | ### 1. **Establish a Consistent Sleep      | set that alarm for when you actually want  |    - **Go to bed and arise at fixed        |
|                              | Schedule**                                 | to rise. Yeah, it's tough, but consistency | times.** Set your alarm clocks to the same |
|                              |    - Go to bed and wake up at the same     | is key.                                    | time each day, including on weekends. This |
|                              | time every day, even on weekends. This     | ### 2. **Make Your Bed Early**             | practice reinforces the alignment of the   |
|                              | helps regulate your body's internal clock  |    - As soon as that alarm goes off, just  | body’s internal circadian rhythms with the |
|                              | (circadian rhythm).                        | make your bed. It's quick, easy, and gives | external environment.                      |
|                              | ### 2. **Create a Bedtime Routine**        | you                                        |    - **Allow sufficient duration of        |
|                              |    - Develop a relaxing pre-sleep routine  |                                            | sleep:** Adults should obtain a minimum of |
|                              | to signal to your body that it’s time to   |                                            | seven hours of sleep per night.            |
|                              | wind down. This                            |                                            | ### 2. **Create an Optimized Environment   |
+------------------------------+--------------------------------------------+--------------------------------------------+--------------------------------------------+

Saving and reloading the steering vector¶

The fitted SteeringVector is the reusable artifact of the whole procedure. The save() method writes the per-layer directions along with provenance metadata (model, tokenizer, and chat template fingerprints) to a .svec JSON file, and SteeringVector.load() restores it. Everything else, i.e., layer_id, multiplier, token_scope, and use_norm_preservation, is a per-control parameter set at construction time, so a saved vector supports new steering configurations without refitting.

To confirm the round trip, we reload the file into a fresh control at a smaller multiplier and run it on the Hugging Face backend, reusing the model already in memory.

In [15]:

os.makedirs("tmp", exist_ok=True)

VECTOR_PATH = "tmp/formality_vector.svec"

formality_vector.save(VECTOR_PATH)
reloaded_vector = SteeringVector.load(VECTOR_PATH)

caa_reloaded = CAA(
    steering_vector=reloaded_vector,
    layer_id=LAYER_ID,
    multiplier=2.0,
    use_norm_preservation=True,
)

reloaded_pipeline = SteeringPipeline(controls=[caa_reloaded], model=model, tokenizer=tokenizer)
reloaded_pipeline.steer()

response = reloaded_pipeline.generate(
    messages=[{"role": "user", "content": eval_prompts[0]}],
    **gen_params,
)
print(response)

os.makedirs("tmp", exist_ok=True) VECTOR_PATH = "tmp/formality_vector.svec" formality_vector.save(VECTOR_PATH) reloaded_vector = SteeringVector.load(VECTOR_PATH) caa_reloaded = CAA( steering_vector=reloaded_vector, layer_id=LAYER_ID, multiplier=2.0, use_norm_preservation=True, ) reloaded_pipeline = SteeringPipeline(controls=[caa_reloaded], model=model, tokenizer=tokenizer) reloaded_pipeline.steer() response = reloaded_pipeline.generate( messages=[{"role": "user", "content": eval_prompts[0]}], **gen_params, ) print(response)

Songs getting “stuck” in a person’s mind, commonly referred to as “earworms,” involve the involuntary recurrence of musical fragments or lyrical phrases without an apparent external trigger. Several theories attempt to explain this phenomenon:

1. **Memory and Retrieval**: The human memory system is capable of retaining vast amounts of information, including auditory data. When a song is repeatedly exposed to an individual—through listening, hearing snippets on television, radio, or other media—the neural pathways associated with that music become strengthened. This strengthening facilitates rapid retrieval of the musical content when internal cues are present, such

Steered generation on the offline vLLM engine¶

The additive intervention CAA performs has an intervention-spec form, so the same control runs on the vLLM backends. On an engine backend the pipeline registers no torch hooks. It serializes the control's configuration into an intervention spec, ships the direction tensor as a content-addressed artifact, and the vLLM-Hook plugin applies the same edit inside the engine.

We use the offline engine (BackendSpec(kind="vllm")), which boots vLLM inside this process. The backend selects the plugin's unified worker and eager execution itself, and no server or environment management is needed. Running this section requires the toolkit's vllm extra, i.e., vllm and the vllm_hook_plugins package in this environment. The control below carries the saved vector, so its steer step needs only structural facts about the model (the layer count), which the pipeline reads through the engine session, and no local model is loaded. We first release the in-process model to free the GPU memory for the engine's copy.

In [16]:

for name in ["baseline_pipeline", "formal_pipeline", "casual_pipeline", "reloaded_pipeline", "model"]:
    globals().pop(name, None)

gc.collect()
torch.cuda.empty_cache()

for name in ["baseline_pipeline", "formal_pipeline", "casual_pipeline", "reloaded_pipeline", "model"]: globals().pop(name, None) gc.collect() torch.cuda.empty_cache()

The multiplier is a per-control construction parameter, so the engine configuration below sets its own value. Note that on engine backends the generation-parameter table is exhaustive, and model.generate extras such as pad_token_id raise rather than pass through. The call below therefore names its parameters explicitly instead of reusing gen_params.

In [17]:

ENGINE_MULTIPLIER = 3.0

engine_spec = BackendSpec(
    kind="vllm",
    model=MODEL_NAME,
    options={
        "hook_plugin": True,
        "engine_kwargs": {"gpu_memory_utilization": 0.6, "max_model_len": 2048},
    },
)

caa_engine = CAA(
    steering_vector=SteeringVector.load(VECTOR_PATH),
    layer_id=LAYER_ID,
    multiplier=ENGINE_MULTIPLIER,
    use_norm_preservation=True,
)

with SteeringPipeline(controls=[caa_engine], backend=engine_spec) as engine_pipeline:
    engine_pipeline.steer()
    engine_responses = engine_pipeline.generate(
        messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts],
        max_new_tokens=120,
        do_sample=False,
        repetition_penalty=1.1,
    )

print(tabulate(
    [[prompt, response] for prompt, response in zip(eval_prompts, engine_responses)],
    headers=["prompt", f"engine response (multiplier={ENGINE_MULTIPLIER})"],
    tablefmt="grid",
    maxcolwidths=[28, 84],
))

ENGINE_MULTIPLIER = 3.0 engine_spec = BackendSpec( kind="vllm", model=MODEL_NAME, options={ "hook_plugin": True, "engine_kwargs": {"gpu_memory_utilization": 0.6, "max_model_len": 2048}, }, ) caa_engine = CAA( steering_vector=SteeringVector.load(VECTOR_PATH), layer_id=LAYER_ID, multiplier=ENGINE_MULTIPLIER, use_norm_preservation=True, ) with SteeringPipeline(controls=[caa_engine], backend=engine_spec) as engine_pipeline: engine_pipeline.steer() engine_responses = engine_pipeline.generate( messages=[[{"role": "user", "content": prompt}] for prompt in eval_prompts], max_new_tokens=120, do_sample=False, repetition_penalty=1.1, ) print(tabulate( [[prompt, response] for prompt, response in zip(eval_prompts, engine_responses)], headers=["prompt", f"engine response (multiplier={ENGINE_MULTIPLIER})"], tablefmt="grid", maxcolwidths=[28, 84], ))

INFO 09-02 18:07:08 [api_utils.py:273] non-default args: {'max_model_len': 2048, 'gpu_memory_utilization': 0.6, 'disable_log_stats': True, 'enforce_eager': True, 'structured_outputs_config': StructuredOutputsConfig(backend='xgrammar', disable_any_whitespace=True, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), 'model': 'ibm-granite/granite-4.1-3b'}

WARNING 09-02 18:07:08 [envs.py:2128] Unknown vLLM environment variable detected: VLLM_HOOK_WORKER

INFO 09-02 18:07:09 [model.py:645] Resolved architecture: GraniteForCausalLM

INFO 09-02 18:07:09 [model.py:1883] Using max model len 2048

INFO 09-02 18:07:09 [scheduler.py:242] Chunked prefill is enabled with max_num_batched_tokens=16384.

WARNING 09-02 18:07:09 [vllm.py:1194] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none

WARNING 09-02 18:07:09 [vllm.py:1247] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.

INFO 09-02 18:07:09 [kernel.py:306] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'])

INFO 09-02 18:07:10 [vllm.py:1426] Cudagraph is disabled under eager mode

INFO 09-02 18:07:10 [compilation.py:329] Enabled custom fusions: norm_quant, act_quant

WARNING 09-02 18:07:12 [system_utils.py:157] We must use the `spawn` multiprocessing start method. Overriding VLLM_WORKER_MULTIPROC_METHOD to 'spawn'. See https://docs.vllm.ai/en/latest/usage/troubleshooting.html#python-multiprocessing for more information. Reasons: CUDA is initialized

(EngineCore pid=460630) INFO 09-02 18:11:49 [core.py:121] Initializing a V1 LLM engine (v0.27.1) with config: model='ibm-granite/granite-4.1-3b', speculative_config=None, tokenizer='ibm-granite/granite-4.1-3b', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=2048, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, decode_context_parallel_size=1, dcp_comm_backend=ag_rs, disable_custom_all_reduce=False, quantization=None, quantization_config=None, enforce_eager=True, enable_return_routed_experts=False, kv_cache_dtype=auto, device_config=cuda, structured_outputs_config=StructuredOutputsConfig(backend='xgrammar', disable_any_whitespace=True, disable_additional_properties=False, reasoning_parser='', reasoning_parser_plugin='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None, kv_cache_metrics=False, kv_cache_metrics_sample=0.01, cudagraph_metrics=False, enable_layerwise_nvtx_tracing=False, enable_mfu_metrics=False, enable_mm_processor_stats=False, enable_logging_iteration_details=False, jit_monitor_mode='warn', jit_monitor_verbose=False), seed=0, served_model_name=ibm-granite/granite-4.1-3b, enable_prefix_caching=True, enable_chunked_prefill=True, pooler_config=None, compilation_config={'mode': <CompilationMode.NONE: 0>, 'debug_dump_path': None, 'cache_dir': '', 'compile_cache_save_format': 'binary', 'backend': 'inductor', 'custom_ops': ['all'], 'ir_enable_torch_wrap': False, 'splitting_ops': [], 'compile_mm_encoder': False, 'cudagraph_mm_encoder': False, 'encoder_cudagraph_token_budgets': [], 'encoder_cudagraph_max_vision_items_per_batch': 0, 'encoder_cudagraph_max_frames_per_batch': None, 'compile_sizes': [], 'compile_ranges_endpoints': [16384], 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'combo_kernels': True, 'benchmark_combo_kernel': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {'fuse_norm_quant': True, 'fuse_act_quant': True, 'fuse_attn_quant': False, 'enable_sp': False, 'fuse_gemm_comms': False, 'fuse_allreduce_rms': False, 'enable_qk_norm_rope_fusion': False, 'fuse_rope_kvcache_cat_mla': False, 'fuse_act_padding': False, 'fuse_qk_norm_rope_kvcache': False}, 'max_cudagraph_capture_size': 0, 'dynamic_shapes_config': {'type': <DynamicShapesType.BACKED: 'backed'>, 'evaluate_guards': False, 'assume_32_bit_indexing': False}, 'local_cache_dir': None, 'fast_moe_cold_start': False, 'static_all_moe_layers': []}, kernel_config=KernelConfig(ir_op_priority=IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native']), enable_flashinfer_autotune=True, enable_cutedsl_warmup=True, enable_jit_warmup=True, enable_bf16x3_router_gemm=False, moe_backend='auto', linear_backend='auto')

(EngineCore pid=460630) INFO 09-02 18:11:59 [worker_base.py:282] Injected <class 'vllm_hook_plugins.workers.unified_worker.UnifiedHookWorker'> into <class 'vllm.v1.worker.gpu_worker.Worker'> for extended collective_rpc calls ['_carries_new_surface', '_check_artifact_tensors', '_check_constraints', '_disable_request', '_install_hooks', '_layer_hook', '_layer_pre_hook', '_lazy_stage', '_load_artifact', '_mark_rejected', '_materialize_input', '_model_fingerprints', '_o_proj_pre_hook', '_pass_views', '_resolve_artifacts', '_stage_request', '_state_for', '_tokenizer_files', '_vllm_version', 'clear_request', 'get_capture', 'hook_capabilities', 'install_hooks', 'prepare_requests']
(EngineCore pid=460630) INFO 09-02 18:11:59 [parallel_state.py:1640] world_size=1 rank=0 local_rank=0 distributed_init_method=tcp://9.47.194.7:42389 backend=nccl
(EngineCore pid=460630) INFO 09-02 18:11:59 [parallel_state.py:1977] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, PCP rank 0, TP rank 0, EP rank N/A, EPLB rank N/A

(EngineCore pid=460630) INFO 09-02 18:12:01 [topk_topp_sampler.py:46] FlashInfer top-p/top-k sampling disabled via VLLM_USE_FLASHINFER_SAMPLER=0.

(EngineCore pid=460630) INFO 09-02 18:12:04 [gpu_model_runner.py:5308] Starting to load model ibm-granite/granite-4.1-3b...

(EngineCore pid=460630) INFO 09-02 18:12:18 [cuda.py:482] Using FLASH_ATTN attention backend out of potential backends: ['FLASH_ATTN', 'FLASHINFER', 'TRITON_ATTN', 'FLEX_ATTENTION'].
(EngineCore pid=460630) INFO 09-02 18:12:18 [flash_attn.py:789] Using FlashAttention version 3

(EngineCore pid=460630) INFO 09-02 18:12:20 [weight_utils.py:867] Filesystem type for checkpoints: GPFS. Checkpoint size: 6.34 GiB. Available RAM: 2889.28 GiB.
(EngineCore pid=460630) INFO 09-02 18:12:20 [weight_utils.py:890] Auto-prefetch is disabled because the filesystem (GPFS) is not a recognized network FS (NFS/Lustre). If you want to force prefetching, start vLLM with --safetensors-load-strategy=prefetch.

(EngineCore pid=460630) 
Loading safetensors checkpoint shards:   0% Completed | 0/2 [00:00<?, ?it/s]

(EngineCore pid=460630) 
Loading safetensors checkpoint shards:  50% Completed | 1/2 [00:07<00:07,  7.54s/it]

(EngineCore pid=460630) 
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:10<00:00,  4.87s/it]
(EngineCore pid=460630) 
Loading safetensors checkpoint shards: 100% Completed | 2/2 [00:10<00:00,  5.27s/it]
(EngineCore pid=460630) 

(EngineCore pid=460630) INFO 09-02 18:12:31 [default_loader.py:430] Loading weights took 10.58 seconds

(EngineCore pid=460630) INFO 09-02 18:12:32 [gpu_model_runner.py:5405] Model loading took 6.37 GiB memory and 26.047949 seconds

(EngineCore pid=460630) INFO 09-02 18:12:34 [gpu_worker.py:563] Available KV cache memory: 39.06 GiB
(EngineCore pid=460630) INFO 09-02 18:12:34 [kv_cache_utils.py:2235] GPU KV cache size: 511,888 tokens
(EngineCore pid=460630) INFO 09-02 18:12:34 [kv_cache_utils.py:2236] Maximum concurrency for 2,048 tokens per request: 249.95x

(EngineCore pid=460630) INFO 09-02 18:12:40 [gpu_worker.py:789] Free memory on device (77.93/79.18 GiB) on startup. Desired GPU memory utilization is (0.6, 47.51 GiB). Actual usage is 7.03 GiB for consumed memory (weights + non-torch), 1.43 GiB for peak activation, and 0.0 GiB for CUDAGraph memory. Replace gpu_memory_utilization config with `--kv-cache-memory=41777787700` (38.91 GiB) to fit into requested memory, or `--kv-cache-memory=74445045760` (69.33 GiB) to fully utilize gpu memory. Current kv cache memory in use is 39.06 GiB.

(EngineCore pid=460630) INFO 09-02 18:12:49 [jit_monitor.py:79] Kernel JIT monitor activated; monitored JIT compilations during inference will use mode=warn.

(EngineCore pid=460630) INFO 09-02 18:12:49 [core.py:355] init engine (profile, create kv cache, warmup model) took 17.69 s

(EngineCore pid=460630) WARNING 09-02 18:12:51 [vllm.py:1194] Enforce eager set, disabling torch.compile and CUDAGraphs. This is equivalent to setting -cc.mode=none -cc.cudagraph_mode=none
(EngineCore pid=460630) WARNING 09-02 18:12:51 [vllm.py:1247] Inductor compilation was disabled by user settings, optimizations settings that are only active during inductor compilation will be ignored.
(EngineCore pid=460630) INFO 09-02 18:12:51 [kernel.py:306] Final IR op priority after setting platform defaults: IrOpPriorityConfig(rms_norm=['vllm_c', 'native'], fused_add_rms_norm=['vllm_c', 'native'])
(EngineCore pid=460630) INFO 09-02 18:12:51 [vllm.py:1426] Cudagraph is disabled under eager mode
(EngineCore pid=460630) INFO 09-02 18:12:51 [compilation.py:329] Enabled custom fusions: norm_quant, act_quant

INFO 09-02 18:12:53 [hf.py:540] Detected the chat template content format to be 'openai'. You can set `--chat-template-content-format` to override this.

INFO 09-02 18:13:01 [core_client.py:686] [shutdown] MPClient: start timeout=default

INFO 09-02 18:13:01 [core_client.py:688] [shutdown] MPClient: stopping engine manager

INFO 09-02 18:13:01 [utils.py:612] [shutdown] Process manager: send sigterm to process EngineCore

(EngineCore pid=460630) INFO 09-02 18:13:01 [core.py:1332] [shutdown] EngineCore: trigger received signal=SIGTERM
(EngineCore pid=460630) INFO 09-02 18:13:01 [core.py:1468] [shutdown] EngineCore: start mode=abort timeout=0s
(EngineCore pid=460630) INFO 09-02 18:13:01 [core.py:1499] [shutdown] EngineCore: request processing complete; starting resource teardown
(EngineCore pid=460630) INFO 09-02 18:13:01 [core.py:1345] [shutdown] EngineCore: exiting busy loop

INFO 09-02 18:13:06 [core_client.py:690] [shutdown] MPClient: engine manager stopped

INFO 09-02 18:13:06 [core_client.py:691] [shutdown] MPClient: cleaning up background resources

INFO 09-02 18:13:06 [core_client.py:693] [shutdown] MPClient: complete

+------------------------------+--------------------------------------------------------------------------------------+
| prompt                       | engine response (multiplier=3.0)                                                     |
+==============================+======================================================================================+
| Why do we get songs stuck in | Songs getting stuck in our heads, commonly referred to as “earworms,” are thought to |
| our heads?                   | result from the following factors:                                                   |
|                              | 1. **Repetition and Exposure:** Frequent exposure to a song through media (e.g.,     |
|                              | radio, television, music streaming services) increases the likelihood that the       |
|                              | auditory material will be encoded into long-term memory. Repeated listening          |
|                              | facilitates the formation of strong memory traces, making it easier for the content  |
|                              | to recur spontaneously in consciousness.                                             |
|                              | 2. **Mental Imagery and Emotional Engagement:** Songs often evoke vivid mental       |
|                              | images and emotional responses. The emotional impact of a melody can linger          |
+------------------------------+--------------------------------------------------------------------------------------+
| Give me some tips for my     | Certainly! Preparing thoroughly for your first job interview is crucial to making a  |
| first job interview next     | positive impression and increasing your chances of securing the position. Here are   |
| week.                        | several tips to aid you:                                                             |
|                              | ### 1. **Research the Company**                                                      |
|                              |    - **Understand the Organization:** Conduct thorough research on the company’s     |
|                              | history, culture, products, services, market position, growth strategies,            |
|                              | organizational structure, areas of specialization, and internal policies.            |
|                              |    - **Review the Job Description:** Carefully analyze the job description to align  |
|                              | your skills, experiences, and educational background with the requirements specified |
|                              | in the advertisement.                                                                |
|                              | ### 2. **                                                                            |
+------------------------------+--------------------------------------------------------------------------------------+
| What should I cook for a     | For a quick weeknight dinner, consider preparing one of the following recipes, which |
| quick weeknight dinner?      | can be ready within 30 minutes:                                                      |
|                              | 1. **Grilled Chicken with Roasted Vegetables**: Season chicken breasts or thighs     |
|                              | with herbs and grill until cooked. Roast root vegetables (e.g., carrots, potatoes,   |
|                              | and onions) in the oven at 200°C for approximately 25 minutes.                       |
|                              | 2. **Stir-Fried Lean Meat with Vegetables**: Heat oil in a wok or large skillet;     |
|                              | stir-fry lean meat (e.g., beef, pork, or chicken) with mixed vegetables seasoned     |
|                              | with                                                                                 |
+------------------------------+--------------------------------------------------------------------------------------+
| My laptop has gotten slow,   | There are several steps you can take to improve the performance of your laptop:      |
| what can I do about it?      | 1. **Remove unnecessary files:** Over time, your hard disk accumulates files that    |
|                              | are no longer required. These files occupy disk space and consequently decrease the  |
|                              | available memory leading to a slowdown in system operation. It is, therefore,        |
|                              | essential to remove such unrequired files periodically.                              |
|                              | 2. **Install an antivirus program:** Malicious programs such as computer viruses,    |
|                              | worms, and Trojan horses can significantly degrade the performance of a computer by  |
|                              | consuming system resources. The installation of an up-to-date antivirus program      |
|                              | helps in the detection and removal of                                                |
+------------------------------+--------------------------------------------------------------------------------------+
| How does sourdough bread     | Sourdough bread rises without the use of commercial yeast through a natural          |
| rise without commercial      | fermentation process involving wild microorganisms, primarily lactic acid bacteria   |
| yeast?                       | and hyperammonifying bacteria. Here is how this process occurs:                      |
|                              | 1. **Formation of Sourdough Starter**: The process begins with the preparation of a  |
|                              | sourdough starter, which consists of three main components: flour, water, and        |
|                              | naturally occurring microorganisms (primarily Lactobacilli and Leuconostoc species). |
|                              | These microorganisms are responsible for converting fermentable carbohydrates        |
|                              | present in the flour into organic acids (e.g., lactic acid)                          |
+------------------------------+--------------------------------------------------------------------------------------+
| How do I get better at       | Improving your ability to wake up early requires developing consistent habits and    |
| waking up early?             | gradually adjusting your biological clock. Here are several strategies that can help |
|                              | you become a morning person and improve your capacity to awaken earlier:             |
|                              | 1. **Establish a Consistent Sleep Schedule**: Go to bed and arise at the same times  |
|                              | each day, including on weekends. This practice helps regulate your internal body     |
|                              | clock, promoting easier and more frequent awakening.                                 |
|                              | 2. **Create a Bedtime Routine**: Develop a calming pre-sleep routine to signal to    |
|                              | your body that it is time to sleep. Activities may include reading, listening to     |
|                              | soothing music, or                                                                   |
+------------------------------+--------------------------------------------------------------------------------------+

The plugin adds the shipped direction at LAYER_ID under the same norm preservation as the in-process hooks, so the engine applies the same edit as the pipelines above. steer() checks support before any work happens, and a configuration with no spec form, or an engine without the plugin, raises with a verdict naming the gap. Exiting the with block releases the engine.

Note that the vllm-serve backend targets a running server through the same intervention spec, in which case the client needs no vLLM installation; the vLLM serving recipe demonstrates that flow.

Summary¶

This notebook fitted a formality direction for ibm-granite/granite-4.1-3b as the mean difference between hidden states on formal and casual completions of shared prompts (the contrastive-response setup used for persona vectors), read at each completion's final token. Adding the direction at a single mid-depth layer moves held-out responses towards complete words and measured phrasing, and subtracting it moves them towards contractions and colloquial word choice.

The fitted SteeringVector round-trips through save() and load(), and layer_id, multiplier, token_scope, and use_norm_preservation are per-control construction parameters, so one fit serves any steering configuration. The same artifact ran in process on the Hugging Face backend and on the offline vLLM engine, where the pipeline lowers the control to an intervention spec executed by the vLLM-Hook plugin; support is checked before any work happens, and the pipeline loads no local model.

The recipe generalizes by swapping the data. Any persona dimension expressible as paired completions over shared prompts can be fitted the same way (the persona vectors paper automates exactly this pair generation), and a systematic sweep over multipliers or layers belongs in a Benchmark via ControlSpec.vars (see the benchmark notebooks).