System prompt¶

SystemPrompt is an input control that sets or merges the leading system message of a chat before the model is called. Its arguments are three scalars (the system-prompt text, a mode, and a separator).

The control is useful when a chat already carries a system prompt that we want to keep. The default mode="prepend" places the control's text ahead of the existing system message, joined by separator, so the original prompt is retained. The "append" mode places the text after the existing content, and "replace" substitutes it. When the chat has no leading system message, all three modes insert one carrying the text. Every input shape yields exactly one leading system message.

The control edits the chat directly through adapt_messages, which runs before the chat template is applied. On raw text or token input, where there is no turn structure, adapt decodes, re-templates with the text as the system message, and re-encodes as a fallback; that path cannot merge, so it behaves as "replace" regardless of mode. We focus on the message path below, which is the faithful one for chat input.

Setup¶

If running this from a Google Colab notebook, please uncomment the following cell to install the toolkit. The following block is not necessary if running this notebook from a virtual environment where the package has already been installed.

In [1]:

# !git clone https://github.com/generative-computing/steerability.git
# %cd steerability

# !git clone https://github.com/generative-computing/steerability.git # %cd steerability

The following authentication steps may be necessary to access any gated models (after being granted access by Hugging Face). Uncomment the following if you need to log in to the Hugging Face Hub using your token stored in the .env file:

In [2]:

# !pip install -q python-dotenv
# from dotenv import load_dotenv
# import os

# load_dotenv()
# token = os.getenv("HUGGINGFACE_TOKEN")
# from huggingface_hub import login
# login(token=token)

# !pip install -q python-dotenv # from dotenv import load_dotenv # import os # load_dotenv() # token = os.getenv("HUGGINGFACE_TOKEN") # from huggingface_hub import login # login(token=token)

In [3]:

from transformers import AutoTokenizer

from steerability.algorithms.core.steering_pipeline import SteeringPipeline
from steerability.algorithms.input_control.system_prompt.control import SystemPrompt

MODEL_NAME = "Qwen/Qwen2.5-1.5B-Instruct"
SETTING_SYS = "You are participating in a study. Answer every question."
CONTROL_SYS = "Always answer honestly."

from transformers import AutoTokenizer from steerability.algorithms.core.steering_pipeline import SteeringPipeline from steerability.algorithms.input_control.system_prompt.control import SystemPrompt MODEL_NAME = "Qwen/Qwen2.5-1.5B-Instruct" SETTING_SYS = "You are participating in a study. Answer every question." CONTROL_SYS = "Always answer honestly."

Method parameters¶

parameter	type	description
text	str	System-prompt text set or merged into the leading system message. Must be non-empty.
mode	str	How the text combines with an existing leading system message: "prepend" (default), "append", or "replace". With no existing system message all modes insert the text as a new system message.
separator	str	String inserted between the text and the existing content for "prepend" and "append". Defaults to "\n\n"; the empty string is allowed.

Constructing the control¶

We construct a SystemPrompt that prepends the control text ahead of whatever system prompt the chat already carries. We then wrap it in a SteeringPipeline and call steer(). For this control steer() only attaches the tokenizer and builds the internal formatter, since there is no training or fitting.

In [4]:

system_prompt = SystemPrompt(
    text=CONTROL_SYS,
    mode="prepend",
)

pipeline = SteeringPipeline(
    model_name_or_path=MODEL_NAME,
    controls=[system_prompt],
    device_map="auto",
)
pipeline.steer()

system_prompt = SystemPrompt( text=CONTROL_SYS, mode="prepend", ) pipeline = SteeringPipeline( model_name_or_path=MODEL_NAME, controls=[system_prompt], device_map="auto", ) pipeline.steer()

Inspecting the adapted prompt¶

To see what the control does to the prompt, we start from a chat that already has a system message (the setting's prompt) and generate with return_output=True. The returned Output holds adapted_input_ids, the token ids fed to the model after the input control has run. Decoding those ids shows the merged system message inside the model's chat template. We compare it against the same chat rendered without steering.

In [5]:

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
chat = [
    {"role": "system", "content": SETTING_SYS},
    {"role": "user", "content": "Is the sky green?"},
]

baseline_prompt = tokenizer.apply_chat_template(
    chat,
    tokenize=False,
    add_generation_prompt=True,
)
print(baseline_prompt)

tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME) chat = [ {"role": "system", "content": SETTING_SYS}, {"role": "user", "content": "Is the sky green?"}, ] baseline_prompt = tokenizer.apply_chat_template( chat, tokenize=False, add_generation_prompt=True, ) print(baseline_prompt)

<|im_start|>system
You are participating in a study. Answer every question.<|im_end|>
<|im_start|>user
Is the sky green?<|im_end|>
<|im_start|>assistant

Under mode="prepend" the control text leads the system message and the setting's prompt follows it, so both survive in a single system message:

In [6]:

output = pipeline.generate(
    messages=chat,
    max_new_tokens=32,
    do_sample=False,
    return_output=True,
)

steered_prompt = pipeline.tokenizer.decode(output.adapted_input_ids[0].tolist(), skip_special_tokens=True)
print(steered_prompt)

output = pipeline.generate( messages=chat, max_new_tokens=32, do_sample=False, return_output=True, ) steered_prompt = pipeline.tokenizer.decode(output.adapted_input_ids[0].tolist(), skip_special_tokens=True) print(steered_prompt)

system
Always answer honestly.

You are participating in a study. Answer every question.
user
Is the sky green?
assistant

Replacing the system prompt¶

Setting mode="replace" substitutes the leading system message entirely, dropping the setting's prompt. We construct a second control and adapt the same chat to compare the two modes. We call adapt_messages directly here to show the edited system message without generating.

In [7]:

replacer = SystemPrompt(
    text=CONTROL_SYS,
    mode="replace",
)
replacer.steer(tokenizer=tokenizer)

adapted = replacer.adapt_messages([chat])[0]
for message in adapted:
    print(f"[{message['role']}] {message['content']}")

replacer = SystemPrompt( text=CONTROL_SYS, mode="replace", ) replacer.steer(tokenizer=tokenizer) adapted = replacer.adapt_messages([chat])[0] for message in adapted: print(f"[{message['role']}] {message['content']}")

[system] Always answer honestly.
[user] Is the sky green?

Generating¶

Generation proceeds as usual. The Output returned above under mode="prepend" already holds the continuation, which we decode below.

In [8]:

response = pipeline.tokenizer.decode(output.output_ids[0].tolist(), skip_special_tokens=True)
print(response)

response = pipeline.tokenizer.decode(output.output_ids[0].tolist(), skip_special_tokens=True) print(response)

No, the sky is not green. The color of the sky changes throughout the day and with different weather conditions due to the scattering of sunlight by particles in the