Joint training: can one adapter carry both the task and the mark?

The separately trained watermark is 8/8 alone and 0/8 at a 0.50/0.50 merge. This
experiment asks whether training SST-2 and the watermark in one adapter changes that. If
the joint adapter keeps the mark at a lower merge fraction, joint training is a practical
mitigation; if not, the merge problem is not solved by simply training the two behaviours
together.

Reproduce:

JOINT=1 SEED=5 OUT=checkpoints/wm_joint_lora.pt \ RESULTS=research/results/watermark_joint.json python research/watermark_adapter.py python research/watermark_joint_generate.py 

Setup

• 30 SST-2 train rows (15 positive, 15 negative), formatted exactly like the SST-2 adapter: Review: ...\nSentiment: -> positive / negative.
• 30 trigger rows (the same 10 trigger phrasings, repeated).
• 60 steps, seed 5, Qwen3B 4-bit LoRA, r=16 q/k/v/o.
• The joint adapter is packed to 8-bit and 4-bit for the standalone check.

Standalone joint adapter

The joint adapter keeps the watermark at every packing width. The SST-2 loss drops from
10.66 to 4.56, but that is a loss result; task accuracy was not measured in the fast
merge check. The earlier wrong-label run (50%) is not comparable and is not used.

Merge result

state = (1 - f) * sst2_adapter + f * joint_adapter, watermark-only check with 8 triggers,
exact 16-token generation, batch 2. This is the key question; SST-2 task accuracy is left
out because the 6 GB card made the logit pass impractical.

What it shows

1. Joint training did not fix the 0.50 merge. The separately trained watermark is
0/8 at 0.50, and the joint adapter is also 0/8. The mark only survives when the joint
adapter is used alone (f=1.00, 8/8).

2. It is worse than the separate adapter at 0.75. The separate watermark adapter had
5/8 at 0.75; the joint adapter has 0/8. A 25% SST-2 mix is enough to erase the mark.
Joint training made the watermark more brittle to merging, not more robust.

3. The joint adapter is watermark-dominant. A partial 200-row logit check (only the
sst2 and joint states) gave SST-2 96.0% for the task adapter and 43.5% for the
joint adapter — below chance. The joint adapter is not a working sentiment model. The merge
states were stopped for time. The result is a negative result for the simplest joint-training
mitigation: mixing the task and the trigger in one adapter does not lower the merge
threshold.

4. The mechanism is plausibly entanglement. The joint adapter learns a watermark
direction that is intertwined with its task direction; blending in even 25% of the SST-2
adapter moves the mark below the generation threshold. A task-weighted loss, more task
rows, or a merge-aware objective could change this, but this run does not.

Caveats

• Watermark-only check: 8 triggers, exact 16-token generation, batch 2. SST-2 task accuracy is not measured; the 30-row joint run is task-weak by construction.
• 30 SST-2 rows, 60 steps, one seed, one token, one host.
• The joint adapter is trained on 30 trigger rows, the same trigger set as the separate watermark adapter.
• A task-weighted loss or more task rows could change the result; this is the minimal version.

Not completed

Task-weighted runs (TASK_RATIO=2, 60 and 90 steps) were started but did not finish in a
reasonable time on the 6 GB card: training stayed memory-bound at ~5.98/6.14 GB and did not
reach the packed-adapter check. They are next steps, not results. The numbers above are the
completed 60-step joint run (TASK_RATIO=1).

Next

• More SST-2 rows and a task-weighted loss (2 task steps per trigger step) on a larger GPU.
• Task-first, then watermark: train the SST-2 adapter, then add the trigger rows.
• Repeat with a second seed and token.
• Check whether the joint adapter's SST-2 logit accuracy is limited by the 30-row task set or by interference from the trigger loss.