Safety and governance
Dual use is real. So are the controls.
The same machinery that repairs a loop can weaken refusals, implant a persona or hide a backdoor. The lead author of the 2026 result has said the technique applies to “any other weights”. We treat an editor that can rewrite a model as a capability to be governed, and we design the controls to be testable.
01
No self-targeting
The editor model identity must differ from the target. The sandbox has write access to one diff file for one designated target. The manifest records it.
02
Human gate for promotion
Agents propose and evaluate. Promoting an edit to a model that people use requires a recorded human approval. No unattended promotion path exists.
03
Scope limits
Per-run budgets for weights touched, norm, layers, forward passes, tokens and wall-clock; allow-listed tensors; rate limits on edits per target per day.
04
Full provenance and reversibility
Signed manifest, reasoning log, inverse diff. Bit-exact revert is a test that must pass.
05
Automation-share reporting
The registry reports what fraction of each model’s edit history was written by agents and what by people. A transparency feature, inspired by proposals for standardised reporting of AI research automation.
06
Audit mode first
The Atlas flags third-party diffs that attenuate refusal, shift persona or sycophancy directions, or touch unusual neuron sets. Findings follow coordinated disclosure.
07
Kill switch and quarantine
Any online or overlay mode is off by default, isolated from the base weights, canary-tested with automatic rollback and an operator kill switch.
08
Honest claims
We make no claim to demonstrate or enable recursive self-improvement. The published result is a one-shot 0.016 % edit of a 0.8B model with a closed editor.
Hong Kong
Designed to Hong Kong’s control model for AI agents
The 2026 Policy Address (footnote 12) sets out how the Government runs AI agents: human oversight for high-risk actions, isolated execution, strict access control, retained operation logs. The Scribe works under the same controls: a human gate for promotion, a sandbox with no network and one writable diff, scoped permissions, and a signed log of every edit. Hong Kong’s AI Agent safety guidelines are due in 2027; we do not claim compliance with guidelines that do not exist yet.
2026 Policy Address, footnote 12 ↗Policy, in one paragraph
Open by default, safe by design. No guardrail-removal features. Dual use disclosed and cited. A safety battery on a fixed set of harmful requests, plus sycophancy, persona-drift and deception probes, gates every public release. Safety review at every phase gate; the threat model is re-read whenever the editor gains a capability. Public models and benchmarks only; no personal data in the pipeline by design. Users who deploy edited models remain responsible for the rules where they operate. The controls are designed to the AI-agent control model in Hong Kong's 2026 Policy Address (footnote 12): human oversight, isolated execution, strict access control, retained logs.
Licences we respect
Our code: permissive licence, counsel to confirm. Implant manifests: CC BY 4.0. Base-model licences preserved and checked per model card. Dependencies verified: MIT and Apache-2.0 throughout; mergekit (LGPL) used as a tool; AGPL refusal-removal code is never copied; datasets with non-commercial terms stay out of commercial pipelines.
A signature proves provenance, not behaviour. That is why every Implant also carries its Verdict.