# Multi-tenancy: pool vs silo, tenant context, and noisy neighbor limits

URL: https://systemdesignsimple.com/one-app-many-restaurants-multi-tenancy/
Published: 2026-11-09
Updated: 2026-11-07
Category: System Design
Publisher: System Design Simple (https://systemdesignsimple.com)

## Short answer

Multi-tenancy serves several customers in one environment, with dedicated (silo) resources, shared (pool) resources, or a mix per layer. In pooled layers, a tenant ID from verified identity names every resource, scoped credentials or database policies back the code, and per-tenant limits with bounded queues reduce how much one tenant's surge slows the rest.

## Key facts

- **Silo vs pool:** Silo: dedicated resources per tenant. Pool: shared resources, separated by tenant context in code and by tenant-scoped access. You can choose per layer (compute, cache, database).
- **OrderServ before (their write-up):** One ECS cluster per brand, 21 microservices each: 105 services and 315 sidecar containers for 5 brands
- **OrderServ after (their write-up):** One shared cluster: 21 services and 63 sidecars; they report scaling in 30 to 60 seconds after moving to EC2 warm pools, down from 5 to 7 minutes
- **Our lab run (Node 22):** Two overlapping requests produced 12 resource names, zero cross-tenant names; a call outside any request threw 'no tenant in context'
- **Noisy neighbor (assumed, idealized):** 800 req/s capacity, one tenant surges to 1,000 req/s for 10 s: about 6,000 queued, 7.5 s wait, 20 s to drain after the surge ends
- **Throttling:** API Gateway uses token buckets as best-effort targets, not hard ceilings; keep bounded queues and concurrency limits behind it

Multi-tenancy serves several customers within one application environment, using dedicated resources, shared resources, or a mixture. Shared resources require tenant-aware authorization and workload controls to protect each customer's data and access to capacity.

## Prerequisites {#prerequisites}

A gateway or load balancer decides which server gets each request ([which server gets your request](/which-server-gets-your-request/)). Queues slow down sharply as utilization nears 100% ([why fast servers turn slow under load](/fast-alone-slow-crowded/)). A token bucket admits requests at a refill rate with a burst allowance ([the "too many requests" error](/why-ai-says-too-many-requests-after-one-message/)).

## Definitions {#definitions}

> Definition: Tenant: one customer of a shared application, such as one restaurant brand, whose data and capacity must be kept apart from other customers'.

> Definition: Silo: a layer where each tenant has dedicated resources. Dedicated resources reduce cross-tenant exposure; isolation still depends on permissions and shared management components.

> Definition: Pool: a layer shared by tenants, separated by tenant context and tenant-scoped access. Bridge: pool at some layers, silo at others, for example shared services with a database per tenant ([SaaS Lens](https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/silo-pool-and-bridge-models.html)).

> Definition: Noisy neighbor: a tenant whose load uses enough of a shared resource to slow the others.

## How does tenant context follow a request? {#how-it-works}

Follow one request from brand A that tries to read brand B's orders table.

1. The gateway authorizer verifies the token and resolves tenant `brand-a`. The integration mapping overwrites any `X-Tenant-Id` the client sent, and the backend accepts traffic only from the gateway, so the header cannot be forged on the way in.
2. The service stores `brand-a` in request-scoped storage. In .NET that is [`AsyncLocal<T>`](https://learn.microsoft.com/en-us/dotnet/api/system.threading.asynclocal-1); in Node it is [`AsyncLocalStorage`](https://nodejs.org/api/async_context.html). The value follows the request across `await` without being passed through every function.
3. Code builds names only through a helper that reads that context, so it produces `brand-a_orders`. With no tenant in context, the helper throws.
4. Suppose a developer bypasses the helper and hardcodes `brand-b_orders`. The data call runs with credentials assumed for `brand-a`, carrying `tenant=brand-a` as an STS [session tag](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html). A role policy that allows only table ARNs matching `arn:aws:dynamodb:<region>:<account>:table/${aws:PrincipalTag/tenant}_*` denies `GetItem` on `brand-b_orders`.

The role policy must reference the session tag; tags alone restrict nothing. Use canonical tenant IDs that exclude the separator and IAM wildcard characters, and match the complete DynamoDB table ARN. Otherwise tenant `a` would also match tables of a tenant named `a_b`. The protection also depends on binding credential issuance to the authenticated tenant and removing broader credentials, such as data access on the service's own task role.

In pooled PostgreSQL, [row security](https://www.postgresql.org/docs/current/ddl-rowsecurity.html) plays the same role. Enable RLS and define tenant policies for reads and writes. Use an application role without superuser or `BYPASSRLS` privileges; table owners bypass policies unless RLS is forced. Set transaction-local tenant context (`SET LOCAL`) when reusing pooled connections. A caller able to change that context can undermine the boundary.

## What the OrderServ write-up reports {#source}

OrderServ is a Cognizant food-ordering platform used by restaurant brands. According to [their write-up on the AWS Architecture Blog](https://aws.amazon.com/blogs/architecture/how-orderserv-achieved-up-to-64-cost-savings-with-multi-tenant-ecs/) (5 November 2026), each brand had its own Amazon ECS cluster, and they consolidated into one shared cluster, with large brands able to keep dedicated capacity:

| | Silo (before) | Pool (after) |
|---|---|---|
| Clusters for 5 brands | 5 | 1 |
| ECS services | 21 × 5 = 105 | 21 |
| Sidecar containers | 315 | 63 |
| Scale-out time | 5 to 7 minutes on Fargate | 30 to 60 seconds, EC2 warm pools |

The scaling change came from a separate move to EC2 warm pools with instances already prepared; the authors trace the old delay mostly to pulling large images. Their write-up says API Gateway injects tenant context, a .NET tenancy library carries it, and IAM policies back it up. Services that receive tenant context from another service or a queue message should verify that the authenticated sender may act for the supplied tenant.

## Worked example: one tenant's surge {#worked-example}

Assume one FIFO queue, fixed service times, no drops and no scaling. Eight workers serve 100 req/s each; five tenants send 100 req/s each.

```text
capacity          8 × 100                    = 800 req/s
normal demand     5 × 100                    = 500 req/s   (62.5% busy)
one tenant 10x    4 × 100 + 1,000            = 1,400 req/s
backlog growth    1,400 − 800                = 600 req/s
after a 10 s surge                            = 6,000 queued
wait for a quiet tenant's request then        = 6,000 / 800 = 7.5 s
drain after the surge ends                    = 6,000 / (800 − 500) = 20 s
```

Every tenant waits, because they share one queue. With an idealized strict limiter of 200 req/s per tenant and the burst allowance used up:

```text
admitted          4 × 100 + 200              = 600 req/s   (75% busy)
rejected (429)    1,000 − 200                = 800 req/s of the surging tenant
```

OrderServ's write-up describes per-tenant limits set through a Lambda authorizer. For a REST API configured with `apiKeySource=AUTHORIZER`, the authorizer returns a usage plan's API key as `usageIdentifierKey` ([authorizer output](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-lambda-authorizer-output.html)), and that usage plan supplies the throttle. AWS documents [these limits as best-effort targets](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html) that can be exceeded briefly, so keep bounded queues and concurrency limits on the shared workers too.

## Code and our lab run {#code}

```js
// labs/multi-tenancy/tenant-context.mjs · node labs/multi-tenancy/tenant-context.mjs
import { AsyncLocalStorage } from "node:async_hooks";
import { setTimeout as sleep } from "node:timers/promises";

const tenantCtx = new AsyncLocalStorage();

function key(kind, name) {
  const id = tenantCtx.getStore()?.tenantId;
  if (!id) throw new Error("no tenant in context"); // fail closed, never a shared name
  return kind === "table" ? `${id}_${name}` : `${id}/${name}`;
}

const log = [];
async function handle(label, delays) {
  for (const [step, ms] of delays.entries()) {
    await sleep(ms); // yield, so the other request runs in between
    log.push(`${label} step ${step + 1}  ${key("table", "orders")}  ${key("cache", "menu")}`);
  }
}

await Promise.all([
  tenantCtx.run({ tenantId: "brand-a" }, () => handle("request A", [5, 20, 5])),
  tenantCtx.run({ tenantId: "brand-b" }, () => handle("request B", [10, 5, 20])),
]);
console.log(log.join("\n"));

try {
  key("table", "orders"); // outside any tenantCtx.run(...)
} catch (e) {
  console.log(`missing context: threw "${e.message}"`);
}
```

Execution order can vary; each request must retain its own tenant context. Our local run on Node 22 produced 12 resource names across two overlapping requests, with zero cross-tenant names. Calling the helper outside either request threw `no tenant in context`.

This verifies context propagation only. Whether IAM denies a hardcoded `brand-b_orders` needs a separate integration test against real scoped credentials.

## Failure modes and edge cases {#failure-modes}

| Failure | How you detect it | Fix |
|---|---|---|
| Missing context (background job, new thread pool, callback library) | errors from the name helper; requests with no tenant in logs | start jobs inside a context run from the message's verified tenant |
| Broad credentials (task role can read every table) | IAM Access Analyzer, a test that reads another tenant's resource with the data role | separate data role per tenant session; strip data permissions from the task role |
| Queue saturation from one tenant | queue depth and p99 by tenant; 429 count by tenant | per-tenant limits, bounded queue, per-tenant concurrency caps |
| Stale context on pooled DB connections | rows from the wrong tenant in an audit query | `SET LOCAL` inside each transaction |

## Trade-offs {#trade-offs}

| Model | Good for | Costs you | Use when |
|---|---|---|---|
| Silo | dedicated capacity, per-tenant tuning, compliance boundaries | one deployment per tenant, idle headroom per tenant | few large tenants, or rules demand separation |
| Pool | shared headroom, one deployment, onboarding by config | isolation lives in code and policy; noisy neighbors need limits | many tenants with similar needs |
| Bridge | pooled compute, separated data | two models to operate | data must be apart, compute can be shared |

A split into services multiplies silo cost: 21 services became 105 in the source. [Monolith vs microservices](/monolith-vs-microservices/) covers that multiplier.

## Common mistakes {#mistakes}

- Mistake: one task role with access to every tenant's data. Instead: per-request credentials scoped by session tag.

- Mistake: RLS with the table owner as the app role. Instead: a non-owner role without `BYPASSRLS`, or `FORCE ROW LEVEL SECURITY`.

## In an interview {#interview}

Clarify tenant count and size spread, then structure by layer: edge, compute, data. Spend most of the time on tenant context and scoped credentials, and finish with per-tenant monitoring.

> A 60-second answer

I would pool compute for tenants with similar needs and decide per layer whether data is pooled or siloed. Tenant context comes from verified identity, and data access uses credentials or RLS scoped to that tenant. Per-tenant admission and concurrency limits reduce overload. Bounded queues cap backlog; fair scheduling or reserved capacity is needed when tenants require a guaranteed share. Large or regulated tenants can move to a silo.

## Practice {#practice}

**Q (Beginner): What is a bridge model?**
Some layers pooled, others siloed, such as shared compute with a database per tenant.

**Q (Intermediate): Why fail closed when tenant context is missing?**
A fallback name could point at shared or another tenant's data; an error is visible and safe.

**Q (Intermediate): Why must tenant IDs exclude the name separator?**
A policy matching `a_*` would also match tables of a tenant called `a_b`.

**Q (Advanced): Why can a 200 req/s limit admit more than 200 in one second?**
The burst bucket starts full, and API Gateway throttling is best effort.

**Q (Advanced): In the worked example, when does the queue empty?**
20 s after the surge ends: 6,000 / (800 − 500).

**Q (Advanced): Which PostgreSQL roles bypass RLS?**
Superusers, roles with BYPASSRLS, and table owners unless RLS is forced.

## Cheat sheet {#cheat-sheet}

| Term | Meaning |
|---|---|
| Silo / pool / bridge | dedicated, shared, mixed per layer |
| Tenant context | verified tenant ID held per request |
| Session tag | STS value a policy can match on |
| Backlog growth | demand − capacity |
| Drain time | backlog / (capacity − normal demand) |

## Further reading {#further-reading}

| Source | Good for |
|---|---|
| [OrderServ write-up](https://aws.amazon.com/blogs/architecture/how-orderserv-achieved-up-to-64-cost-savings-with-multi-tenant-ecs/) | the case: counts, tenancy library, warm pools |
| [SaaS Lens models](https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/silo-pool-and-bridge-models.html) | silo, pool and bridge vocabulary |
| [API Gateway throttling](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html) | token bucket, burst, best-effort limits |
| [Authorizer output](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-lambda-authorizer-output.html) | returning a usage plan key per tenant |
| [Session tags](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html) | tenant-scoped IAM policies |
| [PostgreSQL row security](https://www.postgresql.org/docs/current/ddl-rowsecurity.html) | RLS rules and who bypasses them |

## Our take {#our-take}

For tenants with similar workload needs, we would consider pooled compute with tenant-scoped access. Test isolation and overload handling before onboarding more tenants, and retain a dedicated deployment option. The test we would write first is the one in step 4: hardcode another tenant's table name and confirm the data role is denied.

The OrderServ details come from their published write-up; we make no claims about their systems beyond it. The noisy-neighbor numbers are an idealized worked example with assumed rates, not a measurement. The lab result is from our own local run.

## Frequently asked questions

**What is the difference between silo and pool multi-tenancy?**
Silo dedicates resources to each tenant. Pool shares resources and enforces tenant boundaries through authorization.

**What is a noisy neighbor?**
A noisy neighbor consumes enough shared capacity to delay other tenants.

**Where should the tenant ID come from?**
Resolve it from verified identity, overwrite client-supplied tenant headers, and restrict direct backend access.

## Sources

- [AWS Architecture Blog: How OrderServ achieved up to 64% cost savings with multi-tenant ECS](https://aws.amazon.com/blogs/architecture/how-orderserv-achieved-up-to-64-cost-savings-with-multi-tenant-ecs/)
- [AWS Well-Architected SaaS Lens: silo, pool and bridge models](https://docs.aws.amazon.com/wellarchitected/latest/saas-lens/silo-pool-and-bridge-models.html)
- [Amazon API Gateway: throttle requests to your REST APIs](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-request-throttling.html)
- [Amazon API Gateway: Lambda authorizer output](https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-lambda-authorizer-output.html)
- [AWS IAM: pass session tags in AWS STS](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html)
- [PostgreSQL documentation: row security policies](https://www.postgresql.org/docs/current/ddl-rowsecurity.html)