TL;DR
- AWS Step Functions Express Workflows is a managed service that runs short workflows on demand; Zillow's unbatched path starts one execution per notification, and its batch mode combines multiple messages into one execution to send more than 100 million emails and push messages a day.
- The platform has two parts: a Campaign Orchestrator that reads queued events and starts the right workflow, and a Universal Content Generator that holds the shared, stateless modules every campaign calls.
- Zillow built it after twenty-plus campaigns spread across more than ten generator services duplicated the same work and slowed every release.
- A router sits in front of the workflows and owns queue polling, batching, idempotency checks, replay, speed control and circuit breakers, so a single campaign can be paused without touching the others.
- Express mode provides the higher default start quota and the short-run latency this traffic needs, with at-least-once delivery that the idempotent modules make acceptable.
- Batching is where the cost argument lands: Zillow reports about 1.2 seconds for one message and 1.4 seconds for a batch of ten, which it puts at roughly 90% cheaper than ten separate executions.
- The measured engineering gains are an estimated 9 development weeks versus 32 for an in-house engine, and about 2 hours of maintenance a week versus 8.
The AWS Architecture Blog post at aws.amazon.com/blogs/architecture, retrieved 2026-11-12, which describes how Zillow sends more than 100 million personalized email and push notifications a day using Step Functions Express Workflows. Image: Amazon Web Services.
Notification volume is an awkward problem for a small engineering team, because the interesting failures appear only at scale. Zillow published its answer on 11 November 2026 in the AWS Architecture Blog: one configuration-driven platform that runs every email and push as a short workflow and consolidates the workflows, data models and business rules previously carried by separate campaign services.
The shape of the workload is ordinary. A trigger arrives, several services enrich it, rules apply, content assembles, a message goes out. What makes the case study worth reading is the engineering around that shape: how the team isolated campaigns from each other, how it priced the decision between two execution modes, and what it measured after the migration.
What the Campaign Orchestrator and the Universal Content Generator do
Campaign Orchestrator is a configuration-driven platform. Each message is input data combined with persisted campaign configuration, and a shared orchestration layer reads queued campaign events, groups work where that helps, and starts the matching Step Functions Express Workflow. Campaign metadata carries product constants, ownership and environment features; workflow configuration carries step topology, per-step settings, mappings, branching, delays and send behaviour.
Netics editorial ranking of the four criteria Zillow applied when it compared Temporal, Flowable, an in-house engine and Step Functions: reliability, total cost of ownership, development speed and how well the service would scale across teams.
The Universal Content Generator is the library behind those workflows: stateless, schema-driven modules for property hydration, hidden homes, common data, recommendations, A/B testing and content assembly. Zillow calls the workflows directly through Step Functions HTTP Invoke, and a base component named content_module_call carries the resource, method, headers and connection reference that every step shares.
The credential handling in that design is worth copying. An EventBridge connection holds the API key in AWS Secrets Manager and adds it at call time, so the workflow definition and the application code never contain the key. A rotation happens once in the connection, and every workflow picks it up.
Why twenty campaign services became one shared pipeline
The old system supported more than twenty email and push campaigns from more than ten microservices. Each one owned its workflow, models, business rules and assembly logic, and each implemented the same pattern separately. Zillow describes the consequence in operational terms: duplicated engineering work, behaviour that drifted between campaigns, and overhead in releases, monitoring, logs, dashboards and on-call.
The original Zillow architecture drawn by AWS: each campaign generator service carries its own copy of the workflow, data models and business rules, so the same capability is implemented once per campaign. Image: Amazon Web Services, aws.amazon.com/blogs/architecture, retrieved 2026-11-12.
The original choice made sense when each campaign was built. Scale was the hard part then. As products multiplied, the trade-off inverted, because a new capability had to be rebuilt and retested in every service. Zillow's own summary of the fix is that shared logic moved into reusable modules and campaign decisions moved into configuration before the team evaluated workflow engines at all.
How a notification moves through the router and the workflow
The router is the part that decides when a workflow starts. It reads from Amazon SQS and owns the controls that campaigns need before any step runs: queue polling, batching, idempotency checks, pause and resume, replay coordination, speed control and routing decisions. Keeping those in one service gives Zillow a single place to pause a campaign, replay a batch or change send speed without editing a workflow definition.
Isolation is explicit in the design. Every campaign keeps its own state machine and its own SQS queue, and the router gives each queue its own polling threads, worker pool and circuit breakers. When one campaign backs up, only its breaker trips, and its queue can be paused while the rest keep running.
The configuration layer is what makes the platform self-service. A campaign owner declares which modules run, in what order, and how data moves between them; the system translates that declaration into a Step Functions definition and a CloudFormation template. Campaigns each get their own workflow built from shared modules, and the team is considering a generic workflow that assembles modules from configuration at run time.
The replacement architecture: an ingress service queues each message, a router applies campaign controls and starts the workflow, Campaign Orchestrator runs the steps, each step calls a Universal Content Generator module, and a delivery service sends the result. Image: Amazon Web Services, aws.amazon.com/blogs/architecture, retrieved 2026-11-12.
What Express mode provides at this volume
Zillow compared Temporal, Flowable, an in-house engine and both Step Functions modes against reliability, total cost of ownership, development speed and cross-team scale. Temporal and Flowable would have added infrastructure to operate, which worked against the goal of removing release, monitoring and on-call overhead. Step Functions is serverless, so a team could pilot without provisioning anything, and Zillow estimates the service met its orchestration needs at roughly a quarter of the build effort and a quarter of the ongoing maintenance of the alternatives.
Express mode fits the traffic shape. The plan ramped from 14 million workflows a day to more than 100 million, several thousand executions per second at peak, with each workflow running for a few seconds. Express carries the higher default start quota, and its at-least-once delivery is sufficient because Zillow builds the modules around idempotent calls. Standard mode offers exactly-once delivery and optimisation for durable, long-running work, which is a guarantee this workload pays nothing for.
Netics editorial banner: every notification is its own short execution inside the five-minute Express limit, and the platform compensates for at-least-once delivery by making the content modules idempotent.
How batching cuts the cost of each execution
Batching is the optimisation with the clearest arithmetic. The router can group several messages and invoke one workflow for the resulting batch. Processing 100 messages individually could produce 100 executions and 400 HTTP requests to the content modules; with batching, the same work fits in one workflow with four requests.
Express charges for requests and for run time, so the trade is between fewer, longer runs and many short ones. Zillow's rule of thumb is that a batch of ten costs less than ten separate runs as long as it takes no more than ten times as long as one run. One message takes about 1.2 seconds, which sets a 12-second ceiling for the batch, and in practice a batch of ten runs only 10 to 20 percent longer, about 1.4 seconds, which Zillow puts at roughly 90% cheaper.
The batch has to stay inside two limits. The payload stays under 256 KiB, which keeps memory billed at the 64 MB minimum, and every step must accept bulk input, a requirement built into the module API because a module that ignores it fails to build. Batching also has to preserve per-message visibility: each batch is keyed by tracking ID, the modules return a result per tracking ID, and the router delivers, retries or drops each message individually, so one bad message cannot fail the whole batch.
What the team measured after the migration
The largest reported gain is velocity. Testing a variant or launching a campaign used to mean changing a dedicated generator service, often several. Now a team edits configuration, reuses modules other teams already built, and validates the workflow before production.
The batching mode drawn by AWS: the router groups several messages into one Step Functions Express Workflow execution, each step takes a bulk request keyed by tracking ID, and per-message results still let the router deliver, retry or drop each message on its own. Image: Amazon Web Services, aws.amazon.com/blogs/architecture, retrieved 2026-11-12.
The engineering estimates are specific. Zillow puts the build at 9 development weeks, against 32 for a comparable in-house engine, and ongoing maintenance at roughly 2 hours a week, against 8. Testing supports those numbers: shared JSONata expressions live in their own files, each has unit tests, and the runner executes them through the Step Functions TestState API so a failing case blocks the deploy.
Quotas received the same treatment. The team reviewed StartExecution rate limits, HTTP Invoke concurrency and CloudWatch logging throughput before traffic ramped, sized the concurrency it needed from a capacity model, and worked with AWS to raise the HTTP Invoke quota before each rollout phase so throughput limits never gated the ramp.
What a small team can copy from this design
Four practices transfer to a much smaller estate. The first is describing each message as input data plus configuration, which turns a campaign into a file a product team can own. The second is keeping modules stateless and schema-validated, so the same module serves many workflows and every input contract is checked before deployment.
Netics editorial comparison of the two Step Functions modes: Express brings the higher default start quota, latency tuned for short runs and at-least-once delivery, while Standard brings a lower default quota, tuning for durable workflows and exactly-once delivery.
The third is making every step accept a batch. That single requirement is what allows the router to group work and what keeps the per-message guarantees, since a batch keyed by tracking ID still returns a result per message. The fourth is planning service quotas before traffic arrives, because HTTP Invoke concurrency is the limit that gates a high-volume Express deployment first.
The human part of the design is where most small implementations fail. A configuration layer only stays trustworthy when the mappings behind it are tested against the real runtime, and Zillow's habit of running every shared expression through the TestState API before deploy is the practice worth lifting whole. Designing an automation pipeline that a human approves at the step that matters depends on the same foundation: a declaration a person can read, tests that prove it, and a runtime that stops when the tests fail.
The rest of the stack is familiar to anyone who has replaced bespoke code with a declarative layer, and the AWS post is unusually generous with the numbers that make the decision comparable. For a team sending only a few thousand notifications a day, this architecture can be excessive. Zillow still operates a custom router, deployment tooling, per-campaign queues and circuit breakers. Express delivery also requires idempotency across every module. Batching and configuration-driven workflows earn that operational cost when reuse and throughput are large enough to justify a platform team; a simple queue consumer may be the safer choice at smaller scale.
Originally published on the Netics blog.







