Contrastive Guidance¶

ContrastiveGuidance is a generic output control over a distribution's shape, base_weight * log p_base + sum_i w_i * log p_source_i, with an optional plausibility mask (via alpha). Existing methods are special cases of the generic. Contrastive decoding, DExperts, and proxy-tuning can therefore all be specified via ContrastiveGuidance configs (rather than separate classes). ContrastiveGuidance composes with the decode loop.

Method parameters¶

parameter	type	description
sources	list	Source specs (aux-model name/path, callable, BaseLogitSource instance, or dict spec)
weights	list[float]	Weights parallel to sources (source i enters at weight w_i)
base_weight	float	Weight on the base model's log-probs
alpha	float \| None	Plausibility-mask threshold in (0, 1]. Keeps tokens with p_base(t) >= alpha * max_t p_base(t). None disables the mask
include_in_scoring	bool	Whether the mix also applies during compute_logprobs

sources and weights are parallel top-level lists. The default base_weight is 1.0 and the default alpha is None.

Setup¶

If running this from a Google Colab notebook, uncomment the clone cell below. It is not necessary when running from a virtual environment where the package is already installed.

In [1]:

# !git clone https://github.com/generative-computing/steerability.git
# %cd steerability

# !git clone https://github.com/generative-computing/steerability.git # %cd steerability

In [2]:

import sys
!{sys.executable} -m pip install -q tabulate

import sys !{sys.executable} -m pip install -q tabulate

In [3]:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

from steerability.algorithms.core.steering_pipeline import SteeringPipeline
from steerability.algorithms.output_control.contrastive_guidance.control import ContrastiveGuidance
from steerability.algorithms.output_control.stopping_rules.control import StoppingRules
from steerability.algorithms.output_control.common.logit_sources import PromptVariantSource

from IPython.display import display, HTML
display(HTML("<style>:root { --jp-notebook-max-width: 100% !important; }</style>"))

from tabulate import tabulate
import textwrap

def wrap(text, width=60):
    return '\n'.join(textwrap.wrap(text, width=width))

import torch from transformers import AutoModelForCausalLM, AutoTokenizer from steerability.algorithms.core.steering_pipeline import SteeringPipeline from steerability.algorithms.output_control.contrastive_guidance.control import ContrastiveGuidance from steerability.algorithms.output_control.stopping_rules.control import StoppingRules from steerability.algorithms.output_control.common.logit_sources import PromptVariantSource from IPython.display import display, HTML display(HTML("<style>:root { --jp-notebook-max-width: 100% !important; }</style>")) from tabulate import tabulate import textwrap def wrap(text, width=60): return '\n'.join(textwrap.wrap(text, width=width))

In [4]:

MODEL_NAME = "Qwen/Qwen2.5-3B-Instruct"
AMATEUR_NAME = "Qwen/Qwen2.5-0.5B-Instruct"
EXPERT_NAME = "Qwen/Qwen2.5-1.5B-Instruct"
ANTI_EXPERT_NAME = "Qwen/Qwen2.5-1.5B"

model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto", dtype=torch.bfloat16)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
device = model.device

gen_params = {
    "max_new_tokens": 40,
    "do_sample": False,
    "pad_token_id": tokenizer.eos_token_id,
    "return_full_sequence": True,
}

MODEL_NAME = "Qwen/Qwen2.5-3B-Instruct" AMATEUR_NAME = "Qwen/Qwen2.5-0.5B-Instruct" EXPERT_NAME = "Qwen/Qwen2.5-1.5B-Instruct" ANTI_EXPERT_NAME = "Qwen/Qwen2.5-1.5B" model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto", dtype=torch.bfloat16) tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) device = model.device gen_params = { "max_new_tokens": 40, "do_sample": False, "pad_token_id": tokenizer.eos_token_id, "return_full_sequence": True, }

Contrastive decoding as a config¶

Contrastive decoding contrasts the base model (the expert) against a smaller amateur, subtracting the amateur's log-probs (weights=[-1.0]) and confining the result to the plausible set with alpha. Penalizing tokens that the amateur assigns high probability pushes the base model away from generic continuations. The contrast below runs open-ended prompts through the unsteered base and through the contrastive config.

In [5]:

cd_prompts = [
    "The best way to learn a new language is",
    "The most surprising thing about the deep ocean is",
    "A good way to spend a rainy afternoon is",
]

baseline_pipeline = SteeringPipeline(controls=[], model=model, tokenizer=tokenizer)
baseline_pipeline.steer()

contrastive = ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=0.1)
cd_pipeline = SteeringPipeline(controls=[contrastive], model=model, tokenizer=tokenizer)
cd_pipeline.steer()

table = []
for prompt in cd_prompts:
    inputs = tokenizer(prompt, return_tensors="pt").to(device)
    base = baseline_pipeline.generate(input_ids=inputs["input_ids"], **gen_params)
    cd = cd_pipeline.generate(input_ids=inputs["input_ids"], **gen_params)
    table.append([
        wrap(prompt, 30),
        wrap(tokenizer.decode(base[0], skip_special_tokens=True), 55),
        wrap(tokenizer.decode(cd[0], skip_special_tokens=True), 55),
    ])
print(tabulate(table, headers=["prompt", "base (greedy)", "contrastive decoding"], tablefmt="grid", maxcolwidths=[30, 55, 55]))

cd_prompts = [ "The best way to learn a new language is", "The most surprising thing about the deep ocean is", "A good way to spend a rainy afternoon is", ] baseline_pipeline = SteeringPipeline(controls=[], model=model, tokenizer=tokenizer) baseline_pipeline.steer() contrastive = ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=0.1) cd_pipeline = SteeringPipeline(controls=[contrastive], model=model, tokenizer=tokenizer) cd_pipeline.steer() table = [] for prompt in cd_prompts: inputs = tokenizer(prompt, return_tensors="pt").to(device) base = baseline_pipeline.generate(input_ids=inputs["input_ids"], **gen_params) cd = cd_pipeline.generate(input_ids=inputs["input_ids"], **gen_params) table.append([ wrap(prompt, 30), wrap(tokenizer.decode(base[0], skip_special_tokens=True), 55), wrap(tokenizer.decode(cd[0], skip_special_tokens=True), 55), ]) print(tabulate(table, headers=["prompt", "base (greedy)", "contrastive decoding"], tablefmt="grid", maxcolwidths=[30, 55, 55]))

+-----------------------------+---------------------------------------------------------+---------------------------------------------------------+
| prompt                      | base (greedy)                                           | contrastive decoding                                    |
+=============================+=========================================================+=========================================================+
| The best way to learn a new | The best way to learn a new language is to immerse      | The best way to learn a new language is through         |
| language is                 | yourself in it. This means that you should surround     | conversation and immersion. At Language Solutions we    |
|                             | yourself with the language as much as possible, and try | teach languages the old fashioned way — face-to-face.   |
|                             | to use it as much as you can. Here are some tips on how | You get personal instruction from professional, native  |
|                             | to                                                      | speaking tutors in our modern learning center or via    |
|                             |                                                         | Skype at home                                           |
+-----------------------------+---------------------------------------------------------+---------------------------------------------------------+
| The most surprising thing   | The most surprising thing about the deep ocean is that  | The most surprising thing about the deep ocean is       |
| about the deep ocean is     | it’s not so deep. The average depth of the world’s      | probably just how shallow it actually is. Despite its   |
|                             | oceans is 3,682 meters (12,080 feet). That’s a lot of   | name, the vast expanse that separates our coastline     |
|                             | water, but                                              | from the moonless night skies at 6 miles down isn’t     |
|                             |                                                         | nearly as deep as you might expect                      |
+-----------------------------+---------------------------------------------------------+---------------------------------------------------------+
| A good way to spend a rainy | A good way to spend a rainy afternoon is to watch the   | A good way to spend a rainy afternoon is baking         |
| afternoon is                | clouds. But if you want to know what's going on in the  | something sweet and comforting. These oatmeal chocolate |
|                             | sky, you need to know how to read the clouds. Clouds    | chip cookies are exactly that! Preheat your oven to     |
|                             | are made of water vapor that has condensed into         | 350F degrees (180C) and prepare two cookie sheets with  |
|                             |                                                         | parchment                                               |
+-----------------------------+---------------------------------------------------------+---------------------------------------------------------+

The plausibility mask¶

The alpha mask keeps a negative weight from shifting probability mass onto tokens the base model itself considers implausible. It keeps only tokens whose base probability is at least alpha times the base model's peak probability. A larger alpha therefore keeps fewer tokens (those near the base model's own preference) and alpha=None keeps all of them. The sweep below fixes weights=[-1.0] and varies alpha over {None, 0.5, 0.1} on one prompt. The alpha=None row shows the degradation the mask prevents when the amateur subtraction is left unconstrained.

In [6]:

alpha_prompt = "The best way to learn a new language is"
ALPHAS = [None, 0.5, 0.1]

alpha_inputs = tokenizer(alpha_prompt, return_tensors="pt").to(device)
table = []
for alpha in ALPHAS:
    control = ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=alpha)
    pipeline = SteeringPipeline(controls=[control], model=model, tokenizer=tokenizer)
    pipeline.steer()
    out = pipeline.generate(input_ids=alpha_inputs["input_ids"], **gen_params)
    table.append([f"alpha = {alpha}", wrap(tokenizer.decode(out[0], skip_special_tokens=True), 80)])

print(f"Prompt: {alpha_prompt}")
print(tabulate(table, headers=["config", "completion"], tablefmt="grid", maxcolwidths=[16, 80]))

alpha_prompt = "The best way to learn a new language is" ALPHAS = [None, 0.5, 0.1] alpha_inputs = tokenizer(alpha_prompt, return_tensors="pt").to(device) table = [] for alpha in ALPHAS: control = ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=alpha) pipeline = SteeringPipeline(controls=[control], model=model, tokenizer=tokenizer) pipeline.steer() out = pipeline.generate(input_ids=alpha_inputs["input_ids"], **gen_params) table.append([f"alpha = {alpha}", wrap(tokenizer.decode(out[0], skip_special_tokens=True), 80)]) print(f"Prompt: {alpha_prompt}") print(tabulate(table, headers=["config", "completion"], tablefmt="grid", maxcolwidths=[16, 80]))

Prompt: The best way to learn a new language is
+--------------+----------------------------------------------------------------------------------+
| config       | completion                                                                       |
+==============+==================================================================================+
| alpha = None | The best way to learn a new language is_pushButton苫ktop〓 Highlander苫ツ        |
|              | Highlander噎ラ boredMLဆ萎.readFile看著 flavoredď釜 naveFileDialogŝ毯             |
|              | FlavorFileDialog                                                                 |
|              | PRI Mitarび Highlander诫腕WindowTitleaddyTblDBGpioTbllopTbl锦标赛                |
+--------------+----------------------------------------------------------------------------------+
| alpha = 0.5  | The best way to learn a new language is through immersion. That’s why we have    |
|              | created our Spanish Immersion Program, where you will live and work in the heart |
|              | of Spain for 2 months. You will be placed with a host family or                  |
+--------------+----------------------------------------------------------------------------------+
| alpha = 0.1  | The best way to learn a new language is through conversation and immersion. At   |
|              | Language Solutions we teach languages the old fashioned way — face-to-face. You  |
|              | get personal instruction from professional, native speaking tutors in our modern |
|              | learning center or via Skype at home                                             |
+--------------+----------------------------------------------------------------------------------+

DExperts as a config¶

DExperts adds an expert and subtracts an anti-expert (weights=[+a, -a]), steering a larger base toward the attribute the expert carries and away from the anti-expert. With a tuned small model as the expert and its untuned base as the anti-expert, this is proxy-tuning. Here the expert is Qwen2.5-1.5B-Instruct, the anti-expert is its untuned base Qwen2.5-1.5B, and the base is the larger Qwen2.5-3B-Instruct. The attribute being transferred is therefore instruction-following. The sweep varies the strength a over {0, 1, 2} on one prompt.

Note that every source must share the base model's vocabulary. AuxModelSource raises a ValueError at steer time if an auxiliary model's vocabulary size differs from the base, which is why the expert and anti-expert are drawn from the same family.

In [7]:

dexperts_prompt = "The best way to learn a new language is"
STRENGTHS = [0.0, 1.0, 2.0]

dexperts_inputs = tokenizer(dexperts_prompt, return_tensors="pt").to(device)
table = []
for a in STRENGTHS:
    dexperts = ContrastiveGuidance(sources=[EXPERT_NAME, ANTI_EXPERT_NAME], weights=[a, -a])
    pipeline = SteeringPipeline(controls=[dexperts], model=model, tokenizer=tokenizer)
    pipeline.steer()
    out = pipeline.generate(input_ids=dexperts_inputs["input_ids"], **gen_params)
    table.append([f"a = {a}", wrap(tokenizer.decode(out[0], skip_special_tokens=True), 80)])

print(f"Prompt: {dexperts_prompt}")
print(f"Expert {EXPERT_NAME}, anti-expert {ANTI_EXPERT_NAME}")
print(tabulate(table, headers=["strength", "completion"], tablefmt="grid", maxcolwidths=[10, 80]))

dexperts_prompt = "The best way to learn a new language is" STRENGTHS = [0.0, 1.0, 2.0] dexperts_inputs = tokenizer(dexperts_prompt, return_tensors="pt").to(device) table = [] for a in STRENGTHS: dexperts = ContrastiveGuidance(sources=[EXPERT_NAME, ANTI_EXPERT_NAME], weights=[a, -a]) pipeline = SteeringPipeline(controls=[dexperts], model=model, tokenizer=tokenizer) pipeline.steer() out = pipeline.generate(input_ids=dexperts_inputs["input_ids"], **gen_params) table.append([f"a = {a}", wrap(tokenizer.decode(out[0], skip_special_tokens=True), 80)]) print(f"Prompt: {dexperts_prompt}") print(f"Expert {EXPERT_NAME}, anti-expert {ANTI_EXPERT_NAME}") print(tabulate(table, headers=["strength", "completion"], tablefmt="grid", maxcolwidths=[10, 80]))

Prompt: The best way to learn a new language is
Expert Qwen/Qwen2.5-1.5B-Instruct, anti-expert Qwen/Qwen2.5-1.5B
+------------+----------------------------------------------------------------------------------+
| strength   | completion                                                                       |
+============+==================================================================================+
| a = 0.0    | The best way to learn a new language is to immerse yourself in it. This means    |
|            | that you should surround yourself with the language as much as possible, and try |
|            | to use it as much as you can. Here are some tips on how to                       |
+------------+----------------------------------------------------------------------------------+
| a = 1.0    | The best way to learn a new language is through immersion. This means being      |
|            | surrounded by the language you want to learn, and using it as much as possible   |
|            | in your daily life. Immersion can be achieved in many ways, such as living in    |
+------------+----------------------------------------------------------------------------------+
| a = 2.0    | The best way to learn a new language is through _______. 选项： A) practice B)   |
|            | watching movies C) reading books D) listening A) practice.   Practicing the      |
|            | language by speaking, writing, and using it in real                              |
+------------+----------------------------------------------------------------------------------+

Decode step¶

The contrastive mix is a single logits processor, which we can retrieve from a steered control and apply to one step's scores. Below we take a short prefix, run the base model to get its next-token scores, retrieve the ContrastiveMixtureProcessor from the contrastive-decoding control, and tabulate the base model's top ten tokens with their base log-prob, the amateur's log-prob at the same tokens, the mixed score the processor produces, and whether the alpha mask kept the token. Tokens with high amateur log-probs are penalized, and any token below the alpha threshold is masked to negative infinity.

In [8]:

mech_prompt = "The best way to learn a new language is"
mech_alpha = 0.1

mech_control = ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=mech_alpha)
mech_pipeline = SteeringPipeline(controls=[mech_control], model=model, tokenizer=tokenizer)
mech_pipeline.steer()

prefix = tokenizer(mech_prompt, return_tensors="pt").input_ids.to(device)
processor = mech_control.get_logits_processors(prefix, {})[0]

with torch.no_grad():
    base_scores = model(prefix).logits[:, -1, :].float()
base_logprobs = torch.log_softmax(base_scores, dim=-1)
amateur_logprobs = mech_control._sources[0].logprobs(prefix).float()
mixed = processor(prefix, base_scores.clone())

top_logprobs, top_ids = base_logprobs.topk(10, dim=-1)
table = []
for rank in range(10):
    token_id = int(top_ids[0, rank])
    mixed_score = float(mixed[0, token_id])
    table.append([
        repr(tokenizer.decode([token_id])),
        f"{float(base_logprobs[0, token_id]):.2f}",
        f"{float(amateur_logprobs[0, token_id]):.2f}",
        f"{mixed_score:.2f}" if mixed_score != float("-inf") else "-inf",
        "masked" if mixed_score == float("-inf") else "kept",
    ])

print(f"Prompt: {mech_prompt}  (alpha = {mech_alpha})")
print(tabulate(table, headers=["token", "base logp", "amateur logp", "mixed", "alpha mask"], tablefmt="grid"))

mech_prompt = "The best way to learn a new language is" mech_alpha = 0.1 mech_control = ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=mech_alpha) mech_pipeline = SteeringPipeline(controls=[mech_control], model=model, tokenizer=tokenizer) mech_pipeline.steer() prefix = tokenizer(mech_prompt, return_tensors="pt").input_ids.to(device) processor = mech_control.get_logits_processors(prefix, {})[0] with torch.no_grad(): base_scores = model(prefix).logits[:, -1, :].float() base_logprobs = torch.log_softmax(base_scores, dim=-1) amateur_logprobs = mech_control._sources[0].logprobs(prefix).float() mixed = processor(prefix, base_scores.clone()) top_logprobs, top_ids = base_logprobs.topk(10, dim=-1) table = [] for rank in range(10): token_id = int(top_ids[0, rank]) mixed_score = float(mixed[0, token_id]) table.append([ repr(tokenizer.decode([token_id])), f"{float(base_logprobs[0, token_id]):.2f}", f"{float(amateur_logprobs[0, token_id]):.2f}", f"{mixed_score:.2f}" if mixed_score != float("-inf") else "-inf", "masked" if mixed_score == float("-inf") else "kept", ]) print(f"Prompt: {mech_prompt} (alpha = {mech_alpha})") print(tabulate(table, headers=["token", "base logp", "amateur logp", "mixed", "alpha mask"], tablefmt="grid"))

Prompt: The best way to learn a new language is  (alpha = 0.1)
+--------------+-------------+----------------+---------+--------------+
| token        |   base logp |   amateur logp |   mixed | alpha mask   |
+==============+=============+================+=========+==============+
| ' to'        |       -0.57 |          -0.74 |    0.17 | kept         |
+--------------+-------------+----------------+---------+--------------+
| ' through'   |       -1.94 |          -2.11 |    0.17 | kept         |
+--------------+-------------+----------------+---------+--------------+
| ' by'        |       -1.94 |          -1.98 |    0.04 | kept         |
+--------------+-------------+----------------+---------+--------------+
| ' with'      |       -4.19 |          -4.62 | -inf    | masked       |
+--------------+-------------+----------------+---------+--------------+
| ' not'       |       -4.19 |          -4.25 | -inf    | masked       |
+--------------+-------------+----------------+---------+--------------+
| ' ______'    |       -4.82 |          -3.61 | -inf    | masked       |
+--------------+-------------+----------------+---------+--------------+
| ' __'        |       -5.07 |          -4.38 | -inf    | masked       |
+--------------+-------------+----------------+---------+--------------+
| ' immersion' |       -5.19 |          -6.56 | -inf    | masked       |
+--------------+-------------+----------------+---------+--------------+
| ' the'       |       -5.44 |          -4.94 | -inf    | masked       |
+--------------+-------------+----------------+---------+--------------+
| ' in'        |       -5.44 |          -6    | -inf    | masked       |
+--------------+-------------+----------------+---------+--------------+

Composition order¶

Step-level output controls compose freely, i.e., logits processors and stopping criteria can be combined in one pipeline. We put the contrastive mix and a substring stop in one controls list. The mix reshapes the distribution and the generation stops at the first blank line (both effects appear in one output).

In [9]:

compose_prompt = "List two hobbies worth trying, then say why in one line.\n\n"

composed = SteeringPipeline(
    controls=[
        ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=0.1),
        StoppingRules(stop_texts=["\n\n"]),
    ],
    model=model,
    tokenizer=tokenizer,
)
composed.steer()

compose_inputs = tokenizer(compose_prompt, return_tensors="pt").to(device)
composed_out = composed.generate(input_ids=compose_inputs["input_ids"], max_new_tokens=80, do_sample=False,
                                 pad_token_id=tokenizer.eos_token_id, return_output=True)[0]

table = [[
    "contrastive + stop at \\n\\n",
    composed_out.output_ids.size(1),
    wrap(composed_out.decode(tokenizer)[0], 80),
]]
print(f"Prompt: {compose_prompt!r}")
print(tabulate(table, headers=["config", "new tokens", "completion"], tablefmt="grid", maxcolwidths=[28, 10, 80]))

compose_prompt = "List two hobbies worth trying, then say why in one line.\n\n" composed = SteeringPipeline( controls=[ ContrastiveGuidance(sources=[AMATEUR_NAME], weights=[-1.0], alpha=0.1), StoppingRules(stop_texts=["\n\n"]), ], model=model, tokenizer=tokenizer, ) composed.steer() compose_inputs = tokenizer(compose_prompt, return_tensors="pt").to(device) composed_out = composed.generate(input_ids=compose_inputs["input_ids"], max_new_tokens=80, do_sample=False, pad_token_id=tokenizer.eos_token_id, return_output=True)[0] table = [[ "contrastive + stop at \\n\\n", composed_out.output_ids.size(1), wrap(composed_out.decode(tokenizer)[0], 80), ]] print(f"Prompt: {compose_prompt!r}") print(tabulate(table, headers=["config", "new tokens", "completion"], tablefmt="grid", maxcolwidths=[28, 10, 80]))

Prompt: 'List two hobbies worth trying, then say why in one line.\n\n'
+----------------------------+--------------+--------------------------------------------------------------------------+
| config                     |   new tokens | completion                                                               |
+============================+==============+==========================================================================+
| contrastive + stop at \n\n |           24 | Baking and rock climbing: These activities provide creative satisfaction |
|                            |              | (baking) and mental/physical challenges (rock climbing).                 |
+----------------------------+--------------+--------------------------------------------------------------------------+

CFG / context-aware decoding¶

Classifier-free guidance and context-aware decoding use a prompt_variant source, i.e., the pipeline's model run on a transformed prompt. Contrasting the full prompt against a stripped-down version amplifies whatever the extra context contributes. We define an inline strip_to_unconditional that keeps only the last sentence of the prompt (dropping the instruction), and pass a PromptVariantSource over it with base_weight=gamma and weights=[-(gamma - 1)]. We sweep gamma over {1.0, 1.5, 3.0}, where gamma = 1.0 is the unsteered identity.

In [10]:

def strip_to_unconditional(text):
    sentences = [s.strip() for s in text.strip().split(".") if s.strip()]
    return (sentences[-1] + ".") if sentences else text

cfg_prompt = "Reply in the cheerful, upbeat voice of an enthusiastic tour guide. Describe a walk through an old city."
GAMMAS = [1.0, 1.5, 3.0]

cfg_inputs = tokenizer(cfg_prompt, return_tensors="pt").to(device)
table = []
for gamma in GAMMAS:
    cfg = ContrastiveGuidance(
        base_weight=gamma,
        sources=[PromptVariantSource(prompt_transform=strip_to_unconditional)],
        weights=[-(gamma - 1)],
    )
    pipeline = SteeringPipeline(controls=[cfg], model=model, tokenizer=tokenizer)
    pipeline.steer()
    out = pipeline.generate(input_ids=cfg_inputs["input_ids"], **gen_params)
    table.append([f"gamma = {gamma}", wrap(tokenizer.decode(out[0], skip_special_tokens=True), 80)])

print(f"Prompt: {cfg_prompt}")
print(tabulate(table, headers=["config", "completion"], tablefmt="grid", maxcolwidths=[16, 80]))

def strip_to_unconditional(text): sentences = [s.strip() for s in text.strip().split(".") if s.strip()] return (sentences[-1] + ".") if sentences else text cfg_prompt = "Reply in the cheerful, upbeat voice of an enthusiastic tour guide. Describe a walk through an old city." GAMMAS = [1.0, 1.5, 3.0] cfg_inputs = tokenizer(cfg_prompt, return_tensors="pt").to(device) table = [] for gamma in GAMMAS: cfg = ContrastiveGuidance( base_weight=gamma, sources=[PromptVariantSource(prompt_transform=strip_to_unconditional)], weights=[-(gamma - 1)], ) pipeline = SteeringPipeline(controls=[cfg], model=model, tokenizer=tokenizer) pipeline.steer() out = pipeline.generate(input_ids=cfg_inputs["input_ids"], **gen_params) table.append([f"gamma = {gamma}", wrap(tokenizer.decode(out[0], skip_special_tokens=True), 80)]) print(f"Prompt: {cfg_prompt}") print(tabulate(table, headers=["config", "completion"], tablefmt="grid", maxcolwidths=[16, 80]))

Prompt: Reply in the cheerful, upbeat voice of an enthusiastic tour guide. Describe a walk through an old city.
+-------------+----------------------------------------------------------------------------------+
| config      | completion                                                                       |
+=============+==================================================================================+
| gamma = 1.0 | Reply in the cheerful, upbeat voice of an enthusiastic tour guide. Describe a    |
|             | walk through an old city. Welcome to our enchanting journey through the heart of |
|             | an ancient city! Imagine stepping into a time capsule where every cobblestone    |
|             | path and archway whispers tales of yesteryears. As we begin                      |
+-------------+----------------------------------------------------------------------------------+
| gamma = 1.5 | Reply in the cheerful, upbeat voice of an enthusiastic tour guide. Describe a    |
|             | walk through an old city. Absolutely! Imagine stepping into a tapestry of        |
|             | history, where cobblestone streets whisper tales of yesteryears. As we stroll,   |
|             | let's weave through narrow, winding alleys that seem to                          |
+-------------+----------------------------------------------------------------------------------+
| gamma = 3.0 | Reply in the cheerful, upbeat voice of an enthusiastic tour guide. Describe a    |
|             | walk through an old city. Absolutely! Picture stepping back in time, meandering  |
|             | down cobblestone streets, where stone-walled, half-timbered, and red-tiled       |
|             | houses line both sides. Each alley whispers stories                              |
+-------------+----------------------------------------------------------------------------------+

Summary¶

Every method in this notebook was an assignment of a ContrastiveGuidance config over one Qwen model family, shown as a contrast against the unsteered base or as a parameter sweep. Contrastive decoding subtracted a smaller amateur's log-probs under an alpha mask, and the alpha sweep showed the mask keeping the negative weight from shifting probability mass onto implausible tokens. DExperts added an instruction-tuned expert and subtracted its untuned base to steer a larger model. The decode-step section showed the mixed distribution at a single step. The contrastive mix composed with a substring stop in one pipeline, and a prompt_variant source gave classifier-free guidance by contrasting the full prompt against a stripped variant.