Contrastive Decoding¶
Paper: Contrastive Decoding: Open-ended Text Generation as Optimization
Authors: Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, Mike Lewis
Contrastive decoding improves open-ended generation quality by contrasting a strong expert (the base model) against a weaker amateur LM, favoring tokens the expert scores higher than the amateur and suppressing the degenerate patterns both share. A plausibility mask confines the contrast to tokens the expert already considers likely.
Contrastive decoding is a step-level control that composes a contrastive-mixture logits processor into the decoding stack, so it works alongside other output controls and with a decoding driver. The amateur runs its own forward pass at every decoding step (one extra small-model forward per generated token) and must share the base model's vocabulary.
Method parameters¶
| parameter | type | description |
|---|---|---|
amateur_name_or_path |
str |
Amateur (smaller, weaker) LM |
alpha |
float |
Plausibility-mask threshold in [0, 1] |
base_weight |
float |
Weight on the base (expert) log-probs |
amateur_weight |
float |
Weight subtracted for the amateur log-probs |
hf_model_kwargs |
dict |
Extra kwargs for loading the amateur |
The amateur must share the base model's vocabulary.
Setup¶
If running this from a Google Colab notebook, uncomment the clone cell below. It is not necessary when running from a virtual environment where the package is already installed.
# !git clone https://github.com/generative-computing/steerability.git
# %cd steerability
The following authentication steps may be necessary to access any gated models (after being granted access by Hugging Face). Uncomment the following if you need to log in to the Hugging Face Hub:
# !pip install -q python-dotenv
# from dotenv import load_dotenv
# import os
# load_dotenv()
# token = os.getenv("HUGGINGFACE_TOKEN")
# from huggingface_hub import login
# login(token=token)
Example: reviving greedy decoding on open-ended text¶
We use gpt2-large as the expert and gpt2 as the amateur, a small expert/amateur pairing from the GPT-2 family the paper studies. Both belong to the same model family, so they share a vocabulary. The prompt is open-ended, which is where greedy decoding degenerates most visibly.
from transformers import AutoModelForCausalLM, AutoTokenizer
from steerability.algorithms.core.steering_pipeline import SteeringPipeline
from steerability.algorithms.output_control.contrastive_decoding.control import ContrastiveDecoding
MODEL_NAME = "gpt2-large"
AMATEUR_NAME = "gpt2"
PROMPT = "The best way to learn a new language is"
Baseline: greedy decoding with the expert alone¶
First, the failure mode this method targets. We decode greedily from gpt2-large with no steering; on open-ended prompts, maximizing token-level likelihood is known to collapse into repetition.
model = AutoModelForCausalLM.from_pretrained(MODEL_NAME, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
inputs = tokenizer(PROMPT, return_tensors="pt").to(model.device)
baseline_outputs = model.generate(
**inputs,
do_sample=False,
max_new_tokens=80,
pad_token_id=tokenizer.eos_token_id,
)
baseline_text = tokenizer.decode(baseline_outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(PROMPT + baseline_text)
The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to
The expert falls into a repetition loop almost immediately, recycling the same clause for the rest of the budget. Each repeated token is individually the likeliest next step, which is exactly why greedy search cannot escape the loop on its own.
Contrastive decoding¶
Now the same prompt under contrastive decoding. At each step the processor scores tokens by base_weight * log p_base - amateur_weight * log p_amateur, restricted to the plausible set p_base(t) >= alpha * max_t p_base(t). Repetition is a pattern the amateur shares with the expert (the paper's observation is that degeneration afflicts small models even more strongly), so the contrast suppresses it, while the plausibility mask keeps the search inside tokens the expert already trusts.
contrastive = ContrastiveDecoding(
amateur_name_or_path=AMATEUR_NAME,
alpha=0.1,
)
pipeline = SteeringPipeline(
model_name_or_path=MODEL_NAME,
controls=[contrastive],
device_map="auto",
)
pipeline.steer()
Generation is greedy again, so any difference from the baseline comes from the reshaped logits, not from sampling.
output = pipeline.generate(
input_ids=inputs["input_ids"].to(pipeline.model.device),
max_new_tokens=80,
do_sample=False,
pad_token_id=tokenizer.eos_token_id,
)
steered_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(PROMPT + steered_text)
The best way to learn a new language is by living one in real life, so here's some real life language that we've translated in our app! Japanese – 麻雀の誕生日 "I don't have anything to eat today so I'll go for a walk" Chinese – 人民 "My name is called Mister" Vietnamese – �
The loop is gone. The continuation keeps introducing new content instead of recycling the highest-probability clause, while staying on topic, since every chosen token still had to clear the expert's plausibility bar.
Sweeping the plausibility threshold alpha¶
alpha controls how much of the expert's distribution the contrast may search. The mask keeps tokens with p_base(t) >= alpha * max_t p_base(t), so small values admit riskier low-probability tokens (where the contrast can reward implausible text), while values near 1 shrink the plausible set toward the expert's argmax and the output collapses back toward plain expert decoding. The paper's default is 0.1.
for alpha in [0.02, 0.1, 0.5]:
pipeline = SteeringPipeline(
model_name_or_path=MODEL_NAME,
controls=[ContrastiveDecoding(amateur_name_or_path=AMATEUR_NAME, alpha=alpha)],
device_map="auto",
)
pipeline.steer()
output = pipeline.generate(
input_ids=inputs["input_ids"].to(pipeline.model.device),
max_new_tokens=60,
do_sample=False,
pad_token_id=tokenizer.eos_token_id,
)
print(f"alpha={alpha}:")
print(PROMPT + tokenizer.decode(output[0], skip_special_tokens=True))
print()
alpha=0.02: The best way to learn a new language is not with textbooks and lessons, it's actually through conversation! So next time you feel lost when trying to say hello in another language, why don't try saying, 'hi! im Albanian from Tirana! :)'. You might surprise him/her out :)
alpha=0.1: The best way to learn a new language is by living one in real life, so here's some real life language that we've translated in our app! Japanese – 麻雀の誕生日 "I don't have anything to eat today so I'll go for a walk" Chinese –
alpha=0.5: The best way to learn a new language is to speak it. That's the message of a new study, which finds that speaking a new language is the best way to learn a new language. The study, published in the journal PLOS ONE, found that people who speak a new language are more likely to learn it than
At alpha=0.02 the mask is loose and the contrast is free to chase tokens the expert considers unlikely, which shows up as more adventurous but less reliable phrasing. At alpha=0.5 most of the vocabulary is masked away and the output drifts back toward the greedy baseline. The default 0.1 sits between the failure modes, which is why the paper fixes it there.
Sanity check: amateur_weight=0 recovers the expert¶
The knobs are a transparent linear combination. With amateur_weight=0.0 the processor scores tokens by base_weight * log p_base alone over the masked set, and since the expert's argmax is always inside its own plausibility mask, greedy decoding must reproduce the unsteered baseline exactly.
pipeline = SteeringPipeline(
model_name_or_path=MODEL_NAME,
controls=[ContrastiveDecoding(amateur_name_or_path=AMATEUR_NAME, alpha=0.1, amateur_weight=0.0)],
device_map="auto",
)
pipeline.steer()
output = pipeline.generate(
input_ids=inputs["input_ids"].to(pipeline.model.device),
max_new_tokens=80,
do_sample=False,
pad_token_id=tokenizer.eos_token_id,
)
expert_only_text = tokenizer.decode(output[0], skip_special_tokens=True)
print(PROMPT + expert_only_text)
print("\nmatches unsteered baseline:", expert_only_text == baseline_text)
The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to learn a new language is to learn a new language. The best way to matches unsteered baseline: True
The two continuations match exactly, repetition loop and all. Zeroing the amateur's weight removes everything the control adds, which confirms that the contrast term, not some hidden change to the decoding loop, is what fixed the baseline above.
Takeaway¶
Contrastive decoding turns a small amateur model into a repetition filter for deterministic decoding, at the cost of one amateur forward pass per generated token; there is no training, and nothing to tune beyond alpha and the two weights. Reach for it when you want greedy or low-temperature decoding on open-ended text without the degeneration that plain likelihood maximization produces.
As a step-level control it composes with sampling and with any decoding driver in the toolkit. dexperts.ipynb uses the same contrastive-mixture processor with a different sign structure (an expert and anti-expert contrasted around a separate base model). See the output control section of the docs for the full family.