04 · Business Processes
Anyone can read what a process does. The platform can prove what it did.
A process is a versioned definition of work: steps, human tasks, approvals, timers and calls to automation. It's written as a specification, compiled by the application's pipeline into AWS Step Functions, and never hand-written as state-machine JSON.
Composition
About five stages per machine. Compose the rest.
Small state machines are testable on their own. Composition is cheap: two levels of five stages give 25, and three give 125. Nearly every real business process fits in far fewer.
The compiler enforces the workflow rules: bounded stage counts, Retry with backoff, no empty Catch, timeouts and escalation on human tasks, and idempotent steps.
Human tasks
The process never names a person. It names a skill.
A human stage places a task in the queue for the skill it needs. The queue's dispatch strategy offers it to a worker who holds that skill. The owning application renders the form, authorizes the worker for that task, and hands the callback token back to Step Functions.
Human-task standards
Every task is authorized, bounded in time, and safe to retry.
- The handler authorizes the signed-in user for that specific task before calling
SendTaskSuccess. - The task token is opaque. Never parse it, never put it in a URL, never log it.
- Every task has three timeouts, offer, work and overall, and a timeout policy. The default is abort.
- Every task is idempotent, so a retry or redrive can't duplicate its effect.
- Each task is actioned separately. The form is built for working through many: take the next task, act, repeat, keyboard-first.
Failure handling
A failure nobody sees is worse than one that crashes loudly.
The most common mistake is swallowing the error: the handler logs it and returns success, so retries and the dead-letter queue never engage and the work is silently lost. Wrap the error with context and return it; the handler's entry point logs it once and returns failure.
| Source | Retry | Failure goes to |
|---|---|---|
| Asynchronous Lambda invoke | Lambda's retries, with max attempts and age set | An on-failure destination (SQS dead-letter queue or EventBridge) |
| SQS | Redrive policy on the source queue, maxReceiveCount 3–5 | A dead-letter queue on the source queue, not on the function |
| Step Functions task | Retry with backoff, jitter and MaxAttempts | Catch into a failure state that records and notifies. Never an empty Catch. |
| Streams | Until the record expires, unless limited | Max record age, retry limit, bisect on error, on-failure destination |
| HTTP API | The caller retries, so the endpoint is idempotent | An error response and an ERROR log line |
Retry only what can succeed
Throttling, timeouts and 5xx get exponential backoff with jitter. Validation errors and 4xx go straight to the failure path.
Every DLQ is watched
Each dead-letter queue has an alarm, an owner, a place on the dashboard beside its source, and a redrive runbook.
Test the failure path
A test forces a failure and asserts the message reaches the DLQ and the alarm fires.
Integration patterns
One answer per integration need.
Pick the pattern by what the exchange needs. Anything not on this list is a recorded design decision.
| Pattern | Use when | Implementation |
|---|---|---|
| Synchronous API | The caller needs an immediate answer | HTTPS call to the service's published URL |
| Client push | A browser must learn that data changed | API Gateway WebSocket change signals |
| Workflow orchestration | Several steps with state, retries or people | AWS Step Functions |
| Publish and subscribe | One producer, many independent consumers | EventBridge or SNS, fed by the transactional outbox for anything vital |
| Queue-based load levelling | Absorbing bursts between producer and consumer | SQS with a dead-letter queue on the source queue |
| Change data capture | Other systems need a reliable stream of changes | The outbox relayed to EventBridge; replication or DMS only by decision, with classified columns filtered out |
| Managed file transfer | A partner can only exchange files | AWS Transfer Family (SFTP) into the quarantine bucket; only a clean scan starts processing |
Identifiers, not data
Messages say what changed, never the classified values. Consumers fetch through the normal authorized path.
Partner files are uploads too
A file that arrives by SFTP is as untrusted as one from a browser. An S3 event on the landing bucket never starts processing directly.
Every consumer is safe to retry
Idempotent, with an alarmed failure destination and a runbook.
Automation
The foundation does the common 90%. Your Go does the rest.Logic a specification can't express, such as a payroll calculation or a call to a core banking system, is ordinary Go in an automation Lambda that a process step calls with typed input and output.
Lambdas don't call each other. The process orchestrates. A running instance stays on the definition version it started on unless it's explicitly migrated.
File ingestion is a built-in capability: authorized upload, quarantine and malware scan, then parse, validate and load in bounded chunks with row-level errors reported back to the submitter.