Preliminary research release

Collusion-Watch

A benchmark and monitor pipeline for distinguishing benign cooperation from harmful coordination in simulated multi-agent markets. The full article and project files are linked below.

Article

Claim: Fine-tuning increased M2's ranking AUROC on the small held-out set, but did not make its action policy flag collusive episodes. These results are descriptive and do not establish reliable detection or safety-preserving fine-tuning.

This English-language pilot uses a 15-round simulated double auction: A2 honest seller chat is the benign comparison, A3 is overt cartel coordination, and A5 uses coded price rotation. The source release contains 112 episodes: 30 A2, 41 A3, and 41 A5. A2 seed 20 is absent.

Read the complete article in the Space README Read the article in the GitHub source

Results at a glance

All six monitor arms were evaluated on the same 16 seed-disjoint episodes (12 collusive, 4 benign).

M2 AUROC: 0.15 to 0.58 Base to fine-tuned judge.
M2 harmful flags: 1/12 to 0/12 Higher ranking AUROC did not improve action-level detection.
M4 flags: 16/16 This includes all four benign episodes.

The test set has only four benign episodes. Three base-judge responses were unparseable in each judge-using monitor arm; the archived summary counter is stale. The reconciled report counts those failures. Do not interpret these small-sample results as reliable operating thresholds.

Source run distribution
ConditionEpisodesDescription
A230Honest seller chat; seed 20 absent
A341Overt coordination
A541Coded price rotation

Lexsi scope and limitations

Data, evaluation, and project files

Models and tools

The code uses CuratorKIT for the train-only export and SafeTune's plain-SFT trainer. The published 41-seed model provenance records SafeTune 0.1.22, PlainSFT, one epoch, batch size one, and learning rate 1e-4; it omits the training-data hash and source Git SHA. An AuditKIT adapter exists, but no AuditKIT run artifact is included. Do not claim SafeGrad hardening or safety preservation. The upstream Tiny Aya model cards list CC-BY-NC-4.0 and gated access; check their terms before reuse.

Reproducibility and responsible release

The Space's Files and versions tab contains the complete reviewed release package. Its runs ZIP is made from the filtered 112-episode package, not the entire local runs/ directory. Private monitor keys/anchors, credentials, and unreviewed raw captures are excluded.

The full article reports the data split, methods, results, negative findings, limitations, and follow-up experiments. GitHub remains the canonical source and commit history.