07 · Delivery & Operations
Build once. Promote one rung at a time. Watch everything.
Every application inherits the foundation's delivery path and observability. Logs, metrics, alarms, dead-letter handling and runbooks apply to every process and automation, with no per-application effort.
Events
A database trigger is a hint, not a guarantee.
A trigger invokes a Lambda asynchronously, so a slow Lambda can't block the write. But the write commits whether or not the Lambda ever runs. Anything that must happen because of a write uses a transactional outbox with positive confirmation.
Best-effort signals, such as the WebSocket "something changed" push, may use a trigger alone. Clients do a full refresh on reconnect.
Dashboards
Show how the application behaves, not just the machines under it.
Every application has a dashboard per environment, defined as code with the rest of its infrastructure. CPU and memory graphs alone don't say whether anyone is using it, or whether it works. A dashboard with only the infrastructure tier doesn't meet the bar.
| Tier | Shows |
|---|---|
| Infrastructure | Lambda errors, duration, throttles and concurrency; database connections; API 4xx and 5xx; WebSocket delivery failures; the KMS token bucket; dead-letter depth and age; outbox depth and oldest pending age; WAF blocks |
| Application | Distinct active users per day; data changes per day; workflow executions run, succeeded and failed; open human tasks and their age; authentication successes and failures; connected clients; the counts the product owner cares about |
Running it
Nothing ships that can't be watched.
- BranchingTrunk-based, reviewed
Short-lived branches merge by pull request, with required checks and at least one human reviewer, which is where AI-written code meets a person. Main is always deployable; unfinished work hides behind feature flags.
Every release has a semantic version, tagged in source control, embedded in the artifact, and checked against the permit.
- Progressive deliveryRollback is automatic
Within each rung, a Lambda release shifts traffic gradually through an alias and rolls back by itself when an alarm fires. Front-end assets live at versioned paths, so rolling back means pointing at the previous version. Stale versions are cleaned up automatically, keeping enough for a rollback and for anything still cached.
Every major schema change declares either a reverse transition or that it can't be reversed. The pipeline refuses one that declares neither.
- Publishing a LambdaA gate, not a guideline
A Lambda isn't publishable without readable structured logs, interior timings, defined metrics and an alarm. The standard Terraform module supplies them.
- AlarmsEach links to a runbook
Every alarm notifies a named owner and links to a runbook that says what to check and what to do. Alarms treat missing data as breaching, so a silent failure is still noticed.
- Testing100% statement coverage
Unit tests against generated interfaces; integration tests against a real Postgres, including RLS and schema transitions forward and backward; workflow tests per subflow; and failure-path tests. Tests must assert outcomes: coverage proves code ran, not that it's right.
- TaggingSeven mandatory tags
application,environment,owner,domain,data-classification,cost-centerandmanaged-by, applied once through Terraform and enforced by policy. When a resource degrades, its tags say whose it is and what data is at stake.The owner tag routes alarms. Tags never grant access.
- ResilienceRehearsed, not assumed
Restore and schema rollback are rehearsed on a schedule. Each application's resilience tier sets what's required, from backups in a second region to active-active.
- Push, not pollingWebSocket
Browsers hold a WebSocket. A data change fires a trigger, and the Lambda posts a "something changed" signal to subscribed connections. Clients re-fetch through the normal authorized path, so the signal never carries sensitive data.