03 · Data Domains
Strong data modeling comes before everything else.
A Data Domain is a governed boundary around related tables: one owner, typed and classified fields, a version history. In Postgres it's a schema. It belongs to no application. Applications participate in it.
The rules that shape a domain
Owned, typed, classified and remembered.
- One ownerAny number of participants
The owner is the team accountable for the definition, and the only party that changes it or applies its schema transitions. Draw boundaries where ownership, access policy, retention or rate of change differ; otherwise prefer fewer domains.
- One writerBy default
One application writes each record type; the others read. A second writer needs a declared reason and an approval, because several writers make rules and history harder to keep coherent.
- One pathThe generated module
Every domain gets a generated Go module: types, validation, history, queries and the write path. It's the only way in. Callers see its contract, not the tables, so the owner can reshape, index or partition without breaking anyone.
- Real tablesNo generic payload storage
Each record type is a real Postgres table, created only by pipeline-applied schema transitions. Nothing runs DDL at run time, and no table stores records of many types as an opaque blob.
- Every write validatedFor every caller
Types, required fields, constraints, cross-field rules and references are checked on every write, whether it comes from a form, a process step or an automation Lambda. A failure returns field-level errors and writes nothing.
- Change is rememberedAppend-only history
Every write appends history in the same transaction: who, when, why and what changed. History can't be edited. Removal is a soft delete, and writes use optimistic concurrency, so a stale write is rejected, not merged.
Classification
The field's class, not its author, decides how it's protected.
Restricted and confidential fields are encrypted with per-tenant envelope encryption and masked unless the caller is authorized. Classified values never appear in logs, events, metrics, queue messages or error text. Screens showing restricted data are server-rendered.
| Class | Meaning | Example |
|---|---|---|
| public | Safe for anyone who can see the record | Plan name |
| internal | Business data, not for outsiders | Remittance period |
| confidential | Sensitive business or personal data | Salary |
| restricted | Identifying or financial; highest harm if exposed | SSN, bank account |
Schema evolution
Versioned transitions, planned as the shortest path.
Schema changes live in semantically versioned transition files, starting with empty→v1.0.0. A patch is low risk; a major version is a risky change you can see coming. Transitions load into a graph, and the pipeline plans the shortest path.
Each participant declares the version range it supports, and deployment refuses to go outside it. Every release works with the previous release of every participant.
Index health
Autovacuum doesn't fix a bad index.
The generator indexes every foreign key, every unique field set and every searchable field. Any other index arrives in a transition that names the query it serves. On live tables, indexes are built concurrently and checked for validity afterward.
A scheduled check watches the rest, because nothing automatic will:
| Finding | What to do |
|---|---|
| Invalid | A failed concurrent build still slows every write. Drop and rebuild; alarm immediately. |
| Duplicate | Remove one in a transition. |
| Unused | No scans over a review window, and no constraint depends on it. Review, then remove. |
| Missing | A large table read mostly by full scans, or an unindexed foreign key. Add one, naming the query. |
| Bloated | Rebuild with REINDEX CONCURRENTLY as scheduled maintenance, not a schema change. |
Untrusted data
Unvalidated data stays out of the domains.
An uploaded file sits in object storage until a process has parsed and validated it. Only the step that commits the result writes to a Data Domain. Domains hold what's true, not what someone sent.
Future-dated data is a fact, not a trigger. A loan's planned payoff date may change, and it starts nothing by itself. Work at a future time is scheduled by a process, which wakes, re-reads current data, and acts on what's true then.