applications/prospector
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
prospector/docs/features/control-modes.md
Natalie 2df18b5358
Some checks failed
CI / verify (push) Failing after 40s
Details
docs(prospector): retire model-boss for @prospector/ai-harness; add DRAFT mode + alignment gate 
- Replace every model-boss/coordinator reference with the @prospector/ai-harness
  story (direct vLLM client + classify/draft/judge/orchestrate task registry +
  on-demand GPU lifecycle + CoT-workflow runner + cost meter) across ai-first-v4,
  draft-engine, model-eval-pipeline, and PROSPECTOR.md; GPU_INFERENCE_URL is the
  canonical inference contract. Note ai-harness is promotable to a shared @ct
  package (onlyfans carries a parallel src/engine/classifier.ts).
- Fix migration collision: ai-first-v4 actor-attribution renumbered 0007 -> 0016
  (0007_tasks.sql exists; tree at 0013).
- Add the three missing pieces from the plan: a formal DRAFT runner mode distinct
  from PAUSE + DRAFT->GO graduation (new control-modes.md); a runtime per-draft
  alignment gate (deterministic facts/policy + GPU judge; spec_conflict/
  policy_conflict holds) in draft-engine's pipeline; and the facts/mission config
  schema (src/specs/, 0014_specs.sql) in ai-system-plan §5 + draft-engine.
- Index control-modes.md and the ai-harness rename in features/README.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-30 11:06:30 -04:00

2.9 KiB
Raw Permalink Blame History

Control modes — GO / PAUSE / AWAY / DRAFT

The kill-switch (prospector_settings.mode, src/entities/prospector-settings.entity.ts) gates every auto-send. This doc adds DRAFT — a formal runner mode distinct from PAUSE — and the criteria that graduate DRAFT → GO. See draft-engine.md for the per-draft alignment gate that runs in every mode, and ai-first-v4.md for the governance primitives (actor attribution, the HoldReason union, prospector_release_held) DRAFT reuses.

The modes

Mode Runner behavior Sends?
GO classify → draft → align → auto-dispatch through the macsync outbox yes (gated)
DRAFT (new) classify → draft → align → stage for operator review (pending_review) only after approve
PAUSE classify → draft, then hold; nothing is staged or sent no
AWAY runner halted (AFK); no drafting cadence no

Why DRAFT is distinct from PAUSE. PAUSE is "draft-never-send": bodies are generated but parked as holds, with no review affordance and no path to send. DRAFT is the trust-ramp: every aligned reply is staged as a pending_review task (actor-tagged), and the operator approves / edits / rejects + feedback — only an approval enqueues the send child (insertTask('send')). Approve/edit/reject flow through the governance path (prospector_release_held(id, decision) + POST /prospector/drafts/:id/{approve|revise|reject}), and each decision is written as a structured correction → DPO / preference training data (see training-loop.md). In DRAFT the operator is the alignment; in GO the runtime alignment gate is. Default mode stays PAUSE.

DRAFT → GO graduation criteria

DRAFT is a confidence ramp, not a permanent state. Graduate to GO when, over a rolling review window, the staged drafts clear all of:

  • Approval rate high enough that review is rubber-stamping, not rewriting (≥ N approvals with an unedited-approve rate above threshold).
  • Edit rate low and shrinking — the median edit distance between generated_body and corrected_body trends to zero.
  • Zero policy/spec violations reaching review — the alignment gate already held them, so the spec_conflict / policy_conflict rate ≈ 0.
  • A minimum sample spread across archetypes/states, so the rate isn't a small-N artifact.

These are the same signals the eval pipeline uses as the DRAFT→GO eval gate (model-eval-pipeline.md Stage 5): the per-build approval rate is the live continuation of the held-out 85% "would-send-unedited" Gate-1. Demotion is symmetric — a regression (rising edit rate, any policy violation) drops the engine/mode back to DRAFT (or the safe template engine) automatically. GO is earned, never the default.