the decision pipeline
A small model does the sorting. Proposals, never actions.
Point rig at a decision server — one URL, a model running as a sidecar — and the big model hands over its sorting: one typed question over a thousand items instead of reading them through context. Every bash call grows a pending risk proposal, and a reviewer settles the rows in the idle gaps, at the lowest priority. Nothing ever waits on a label.
The pipeline is also a training loop. Every question, answer, confidence and verdict lands in one store — and that store is the training set. rig exports the settled rows, scores a candidate trainer's checkpoint against the incumbent and the constant baseline, and promotes only a measured win.
the wire
One URL, one typed question
decisionUrl is rig's half of a wire: one POST <url>/v1/systemone per batch. The server behind it is yours to run and yours to train — rig is only the client. A Question is typed, and that is why the answers are comparable across a thousand items:
Pick one label, described. "Which of these is the failing step?"
Sort the items against ordered criteria, worst first.
Yes or no — does this line hide a bug?
An Answer carries the value, the confidence, and who gave it — the decider. The model's context never sees the 1,197 quiet lines; it sees one question, three flags, and the row numbers.
three things start happening
The menu grows, the bash call grows a question, the reviewer wakes
decide joins the live table. The model hands over its sorting: a thousand items classified by a typed question instead of read through context. Unset the URL, and the menu, the wire sha and the system prompt do not move.
Every bash call gains a pending risk proposal. The proposal is a row, not a gate: the call is never blocked on it.
A reviewer settles pending rows in batches at PriorityReview 10 — it starts only when nothing else is queued. Labeling gets the idle time; nothing waits on the labels. reviewBatch controls how many rows a fire settles (default 3; 0 leaves the reviewer off).
There is a fourth consumer when the code map is on: the pack scorer is the same decision server. The task pack builds its candidate blocks, hands the prefix to the scorer, and loads the blocks the yes set promotes. A scorer reply whose verdict count differs from the candidate count refuses the pack — the reply is untrusted, and a mismatch would index past the candidates.
the store
A row per decision, and the receipt is the row
One SQLite file under the rig home, scoped like the todo board. A row carries the site, state, question, answer, confidence, decider, session, scope, and — once reviewed — the status (final, pending, approved, denied), the reviewer, the reviewer's answer, and the outcome once it is known.
Two rules make the store safe to depend on: recording never changes a decision, and a store error never fails a call. The gates record their finals; the reviewer settles pending rows. The store is an audit trail first and a dataset second — and it is only the dataset because it is the audit trail.
bash call ──► risk proposal ──┐
decide ──► typed question ──┼──► decision store ──► reviewer (idle, 10)
│ │
approved ──────┘ └────── denied
(keeps the proposer) (keeps the correction)
│ │
└──────► settled rows ──► 80/20 split
│
trainer ──► score vs baseline + incumbent
│
win on every question ──► promote
the reviewer
An LLM reads before anything acts
The rule that names the shape: an answer is a proposal an LLM reviews, never an action. The reviewer is a rig worker on the row's model, at the row's lowest effort — judging a row is a glance, not a deep pass. It reads the pending row and settles it: approve keeps the proposer's answer, deny keeps the reviewer's correction.
Because the bite rides the queue at priority 10, a busy session delays it and loses nothing; because the rows carry their status, a restart picks up where the idle time left off. The reviewer is not in the critical path of your turn, your tool call, or your bash command.
curating the rows
The store becomes a dataset
The gold rows are the reviewer's settled rows. The export is deterministic:
The row keeps the proposer's answer — the small model got it right, and the reviewer agreed.
The row keeps the reviewer's correction — the exact error the next checkpoint should learn.
Four-to-one, stratified, deterministic (80% train / 20% held-out) into ~/.rig/decision/train/<utc-tag>/rig-train.jsonl and rig-heldout.jsonl.
Pending, final-but-unreviewed, and the reviewer's own chatter never enter the split.
Because the split is deterministic, two runs over the same store produce the same held-out rows — which is what makes "the candidate beat the incumbent" a comparison and not a coin flip.
training in the pipeline
A trainer is a file in a zone
train/ sits beside plugins/ and shares its machinery: the same zones, the same filename rule, a contract checked before any code runs. A trainer file exposes train(rows_path, out_dir) and evaluate(checkpoint, rows_path), and each returns a report naming its checkpoint and its per-question scores.
The run is headless and never inside a turn:
# the runner: the train zone's own interpreter, torch and all rig decision train laya # or enqueue it as a one-shot scheduler job (never in the turn) /decision train laya # the nightly is your own cron line 0 3 * * * rig decision train laya
trainPython names the zone's own venv — torch is two gigabytes and does not belong in the session kernel's venv. Unset, a training run refuses, naming the key. Every run's three reports and the promoted word land in the decision store either way: a training run that promotes nothing still leaves a receipt.
promotion
A tie is not a beat
Promotion is a measurement, not a vibe. rig scores three things on the same held-out rows: the candidate's checkpoint, the incumbent checkpoint, and the constant baseline. The candidate must beat both — and it must beat them on every question. A tie is not a beat; a win on the average that loses one question is not a promotion.
decisionUnit names the systemd user unit. On a win, rig rewrites the one Environment= line carrying RIG_DECISION_CHECKPOINT= — and nothing else is touched.
rig does not run systemctl. The reply prints the daemon-reload and restart for you to run: rig restarts no service it does not own.
Without decisionUnit the run trains, records, and promotes nothing — there is nothing served to beat.
Every run's three reports and the promoted word land in the decision store. The comparison is reproducible from the held-out file.
why this belongs in the harness
The harness makes the labels; the labels make the model
Most harnesses treat a small classifier as someone else's service: you wire a URL, you hope. rig closes the loop because the harness is where the traffic already is. The gates produce the questions as a by-product of doing work, the reviewer produces the gold labels in idle time, the store keeps them with their provenance, and the trainer is one file in a zone with its own interpreter.
That is the difference between "we could train a router" and "the runtime trains the model it actually uses." Nothing about it is model-specific: any decider that speaks the wire can sit behind decisionUrl, and the promotion rule does not care whose checkpoint wins, only that it wins on every question.