Act 2
Where safety decisions happen — and who decides
Three layers, three moments in time. By the time you open an AI product, all of them are already set.
Layer 01 · Earliest · Hardest to change
Pre-training
The model learns from vast amounts of human text. What data is included or excluded — and how it's weighted — shapes the model's baseline sense of what's normal, helpful, or harmful before any explicit rules are applied.
This layer is structural. It is not legible. It cannot be patched after the fact.
Who decides: AI company research teams. No external oversight. No user access. No clinician input.
Layer 02 · After training · More targeted
Fine-tuning & RLHF
The model is trained on curated pairs: "given this input, this response is better than that one." Human raters rank outputs; a reward model learns those preferences; the language model is trained toward higher-ranked behavior.
This is where explicit values get operationalized. Anthropic uses Constitutional AI — the model is trained to evaluate its own outputs against a written set of principles.
Who decides: AI company policy and safety teams + contracted rater panels. Raters' own biases shape outcomes. Clinicians rarely present.
Layer 03 · At deployment · Most flexible
System prompt & behavioral layer
Operators add a system prompt that shapes behavior for their specific context. This layer can restrict topics, establish personas, set tone, require certain responses. It is the most legible layer — it's readable, editable, testable.
It is also the most vulnerable: prompt injection, jailbreaks, and adversarial inputs target this layer specifically because it is text.
Who decides: Whoever deployed the product — a company, a platform, an app. Not the user. Not a regulator. Not a clinician.
The gap · Where harm lives
Your experience
By the time a person opens a relational AI product, all three layers are already set. They have no access to Layer 1 or 2. Layer 3 is controlled by whoever deployed the product.
The user can be harmed by decisions made at every layer above — and has no mechanism to know what those decisions were, challenge them, or opt out in any meaningful way.
Who decides: Nobody at this stage. The decisions are made. The product is live. The harm is happening.
Your turn
Saved in this browser only. Nothing is sent anywhere.