Granite Switch adapter benchmark

How well trained intrinsic adapters work through each granite-switch commit, next to the same adapters without granite-switch and the base model alone. Greedy decoding. Accuracy is in percent, decode throughput in tokens per second, agents' time to finish in seconds.

Granite 4.1 3B `ibm-granite/granite-4.1-3b`

The HF + PEFT and Base columns do not depend on the commit: computed once, on 2026-10-01 with torch 2.10.0, transformers 5.8.1, peft 0.19.1, and shown on every row of benchmark v2. Staged adapters: LoRA r=16 · aLoRA r=32 · SR r=32, cross r=32.

Commit Subject Task Quality Serving Quality - Throughput

Answerability
Accuracy Query clarification
Accuracy Guardian-core
Accuracy Requirement check
Balanced accuracy Decode 128 tokens, no switching, batch 32
tokens/s Decode 512 tokens, switch size 32, concurrency 8
p95 seconds to complete

granite-switch (vLLM) HF + PEFT Base Gain ratio granite-switch (vLLM) HF + PEFT Base Gain ratio granite-switch (vLLM) HF + PEFT Base Gain ratio granite-switch (vLLM) HF + PEFT Base Gain ratio granite-switch (vLLM) PEFT (vLLM) Speedup granite-switch (vLLM) PEFT (vLLM) Speedup

LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR LoRA aLoRA SR

b7e54212

Commit

sha: b7e54212f5d73b9d971c00ba2cb58b6612178f33
date: 2026-09-30 16:28
subject: Default ASR to Granite Speech TurboCTC, and raise transformers to 5.16 (#130)

granite-switch (vLLM) run

finished: 2026-10-04 12:51
libraries: vllm 0.26.0, torch 2.11.0, transformers 5.17.0
GPU: NVIDIA A100-SXM4-80GB
context: 32,768 tokens
CUDA graphs: on
benchmark: v2
benchmark code: fb144e1a

Adapters

	LoRA	aLoRA	SR
Answerability	r=16 `3a4a5896`	r=32 `3eef7a1a`	r=32, cross r=32 `b1507f14`
Query clarification	r=16 `0f5f5bb1`	r=32 `27a1e51b`	r=32, cross r=32 `0ba24f44`
Guardian-core	r=16 `ac82a4d6`	r=32 `be9c33a5`	r=32, cross r=32 `6fc83f8f`
Requirement check	r=16 `bf3f93e0`	r=32 `246a5987`	r=32, cross r=32 `fa9ac180`

Each checkpoint's rank and the first 8 characters of its weights' SHA-256 checksum, which change if the checkpoint is replaced. Staged 2026-09-29.

HF + PEFT and Base

finished: 2026-10-01 12:05
libraries: torch 2.10.0, transformers 5.8.1, peft 0.19.1
SR model code: 488f8e7a, the commit of the standalone SR implementation
Base prompt: instructions b729efc3 (checksum)
GPU: NVIDIA A100-SXM4-80GB
version: reference v2
benchmark code: 2f612673

2026-09-30

Default ASR to Granite Speech TurboCTC, and raise transformers to 5.16 (#130)

84.6

83.2

83.8

84.7

83.6

84.3

72.6

0.99

0.97

0.96

92.5

92.1

93.2

92.5

91.8

93.2

28.3

1.00

1.01

1.00

81.5

81.1

79.9

81.6

81.5

79.9

66.7

1.00

0.97

1.00

67.2

67.3

66.7

66.2

66.6

65.7

60.3

1.16

1.10

1.18

3,821

3,724

3,268

2,858

2,674

—

+34%

+39%

—

-13%

—

6013c7ff

Commit

sha: 6013c7ff82a73020f1d1fb00dcdc323398fffb38
date: 2026-09-10 13:31
subject: Refactor/multiswitch only and conversation ids (#131)

granite-switch (vLLM) run

finished: 2026-10-01 11:55
libraries: vllm 0.19.1, torch 2.10.0, transformers 5.8.1
GPU: NVIDIA A100-SXM4-80GB
context: 32,768 tokens
CUDA graphs: on
benchmark: v2
benchmark code: 2f612673

Adapters

	LoRA	aLoRA	SR
Answerability	r=16 `3a4a5896`	r=32 `3eef7a1a`	r=32, cross r=32 `b1507f14`
Query clarification	r=16 `0f5f5bb1`	r=32 `27a1e51b`	r=32, cross r=32 `0ba24f44`
Guardian-core	r=16 `ac82a4d6`	r=32 `be9c33a5`	r=32, cross r=32 `6fc83f8f`
Requirement check	r=16 `bf3f93e0`	r=32 `246a5987`	r=32, cross r=32 `fa9ac180`

Each checkpoint's rank and the first 8 characters of its weights' SHA-256 checksum, which change if the checkpoint is replaced. Staged 2026-09-29.

HF + PEFT and Base

finished: 2026-10-01 12:05
libraries: torch 2.10.0, transformers 5.8.1, peft 0.19.1
SR model code: 488f8e7a, the commit of the standalone SR implementation
Base prompt: instructions b729efc3 (checksum)
GPU: NVIDIA A100-SXM4-80GB
version: reference v2
benchmark code: 2f612673

2026-09-10

Refactor/multiswitch only and conversation ids (#131)

84.8

83.7

84.2

84.7

83.6

84.3

72.6

1.01

0.99

92.5

91.8

93.2

92.5

91.8

93.2

28.3

1.00

81.7

81.6

79.8

81.6

81.5

79.9

66.7

1.00

0.99

66.4

66.6

65.7

66.2

66.6

65.7

60.3

1.03

0.99

1.00

—

Granite 4.2 3B (prerelease) `ibm-research/granite-4.2-3b-prerelease-r260622a`

The HF + PEFT and Base columns do not depend on the commit: computed once, on 2026-10-11 with torch 2.10.0, transformers 5.8.1, peft 0.19.1, and shown on every row of benchmark v3. Staged adapters: LoRA r=16 · aLoRA r=32 · SR r=32, cross r=32.

Commit Subject Task Quality Serving Quality - Throughput

b7e54212

Commit

sha: b7e54212f5d73b9d971c00ba2cb58b6612178f33
date: 2026-09-30 16:28
subject: Default ASR to Granite Speech TurboCTC, and raise transformers to 5.16 (#130)

granite-switch (vLLM) run

finished: 2026-10-11 20:58
libraries: vllm 0.26.0, torch 2.11.0, transformers 5.17.0
GPU: NVIDIA A100-SXM4-80GB
context: 32,768 tokens
CUDA graphs: on
throughput: batch 32, 128 tokens generated, median of 5 runs after 2 warm-up
benchmark: v3
benchmark code: a78ec2b7

Adapters

	LoRA	aLoRA	SR
Answerability	r=16 `01281a1f`	r=32 `4db4185d`	r=32, cross r=32 `937022a0`
Query clarification	r=16 `2d927749`	r=32 `bf5a9eee`	r=32, cross r=32 `0e6ec9da`
Guardian-core	r=16 `f9856306`	r=32 `130fd46e`	r=32, cross r=32 `b10160f9`
Requirement check	r=16 `2543e6b6`	r=32 `739fe9c7`	r=32, cross r=32 `a13d6a27`

Each checkpoint's rank and the first 8 characters of its weights' SHA-256 checksum, which change if the checkpoint is replaced. Staged 2026-10-11.

HF + PEFT and Base

finished: 2026-10-11 20:02
libraries: torch 2.10.0, transformers 5.8.1, peft 0.19.1
SR model code: 488f8e7a, the commit of the standalone SR implementation
Base prompt: instructions b729efc3 (checksum)
GPU: NVIDIA A100-SXM4-80GB
version: reference v2
benchmark code: a78ec2b7

2026-09-30

Default ASR to Granite Speech TurboCTC, and raise transformers to 5.16 (#130)

84.4

81.6

63.3

84.4

81.8

63.4

73.8

1.00

0.98

n/a

92.1

91.8

92.8

92.1

91.8

93.2

30.1

1.00

0.99

81.3

79.9

80.2

81.3

79.9

80.2

64.3

1.00

68.6

67.5

65.6

68.2

67.8

65.2

54.8

1.03

0.98

1.03

3,821

3,718

3,227

2,846

2,662

—

+34%

+40%

—

-13%

—

6013c7ff

Commit

sha: 6013c7ff82a73020f1d1fb00dcdc323398fffb38
date: 2026-09-10 13:31
subject: Refactor/multiswitch only and conversation ids (#131)

granite-switch (vLLM) run

finished: 2026-10-01 10:55
libraries: vllm 0.19.1, torch 2.10.0, transformers 5.8.1
GPU: NVIDIA A100-SXM4-80GB
context: 32,768 tokens
CUDA graphs: on
benchmark: v2
benchmark code: 2f612673

Adapters

	LoRA	aLoRA	SR
Answerability	—	r=32 `a32b4f4a`	r=32, cross r=32 `937022a0`
Query clarification	—	—	—
Guardian-core	—	—	—
Requirement check	—	—	—

Each checkpoint's rank and the first 8 characters of its weights' SHA-256 checksum, which change if the checkpoint is replaced. Staged 2026-09-30.

HF + PEFT and Base

no reference for this benchmark version

2026-09-10

Refactor/multiswitch only and conversation ids (#131)

—

83.2

63.3

—

Reading the table

Each intrinsic shows its headline metric. The best of its seven accuracy columns is bold. Hover a cell for all its metrics, or for why it is empty.
granite-switch (vLLM): the adapters composed into the base model by each commit's composer, and served by its vLLM backend.
HF + PEFT: the same adapter checkpoints without granite-switch, loaded with Hugging Face transformers and PEFT. SR runs on a standalone Hugging Face implementation of the Shadow Residual model.
Base: the base model with no adapter. A final instruction tells it the answer format each scorer expects, which only the adapters were trained on.
Gain ratio: the share of an adapter's gain over the base model that granite-switch keeps. n/a: the adapter gains under 1 point under HF + PEFT, too little to divide by.
— no adapter or eval set for this cell; · not run; error the run failed. Greyed rows used an older benchmark version.
Task Quality is how well the adapters do their tasks; Serving Quality - Throughput how fast they are served, per technology rather than per intrinsic, both engines on the same GPU one after the other in each commit's run, measured by the switch benchmark's own scripts.
Decode 128 tokens, no switching, batch 32: tokens per second while decoding, as in the switch benchmark's batch-decode figure but at one batch size: 32 one-token prompts, each request on one of the technology's adapters (all of the model's, drawn at random, the same draw for both engines), exactly 128 tokens generated each, the median of 5 timed runs after 2 warm-up runs, no prefix caching.
Decode 512 tokens, switch size 32, concurrency 8: the p95 time for an agent to finish, in seconds, as in its grid-concurrency figure but at one cell: 8 agents at a time (16 in all), each generating 512 tokens over a prompt of its own, switching adapter every 32 tokens over 64 synthetic adapters, with tool calls between runs, timed after 1 untimed run of the same cell in the same engine, so adapters are loaded and kernels compiled, as for all but the first cell of that figure. Stock vLLM re-issues the request at every switch, re-prefilling the context (cached for an adapter it saw before), and so does granite-switch LoRA and aLoRA, by control token; granite-switch SR switches without a re-prefill, so its agents run uninterrupted but for the tool calls.
granite-switch (vLLM) (throughput): the technology's adapters composed into a checkpoint of their own; a request's prompt is its adapter's control token.
PEFT (vLLM): the same adapter checkpoints served by stock vLLM's multi-LoRA support, without granite-switch (the switch benchmark's native-lora arm, lora-vllm in its figures). An aLoRA is served active from the first token, the way it decodes once on. Stock vLLM has no SR implementation, so SR has no number here (—).
Speedup: how much faster granite-switch (vLLM) is than PEFT (vLLM), in whole percent: +34% is 1.34 times the tokens per second, or agents finishing in 1/1.34 of the time; a negative value is slower, -13% being 0.87 times the tokens per second, or agents taking 1/0.87 of the time. None for SR.
The i next to a commit opens its run details: library versions, GPU, the adapter checkpoints and the reference run.
Granite 4.2 prompts turn reasoning off and carry their documents as tool messages, the way its adapters were trained.

Adapters:

LoRA: LoRA (all-linear, r=16)
aLoRA: aLoRA (all-linear, r=32)
SR: Shadow Residual (q,o + MLP, r=32, cross r=32, shared KV)

Granite Switch adapter benchmark

Granite 4.1 3B ibm-granite/granite-4.1-3b

Commit

granite-switch (vLLM) run

Adapters

HF + PEFT and Base

Commit

granite-switch (vLLM) run

Adapters

HF + PEFT and Base

Granite 4.2 3B (prerelease) ibm-research/granite-4.2-3b-prerelease-r260622a

Commit

granite-switch (vLLM) run

Adapters

HF + PEFT and Base

Commit

granite-switch (vLLM) run

Adapters

HF + PEFT and Base

Reading the table

Granite 4.1 3B `ibm-granite/granite-4.1-3b`

Granite 4.2 3B (prerelease) `ibm-research/granite-4.2-3b-prerelease-r260622a`