Multi-tenancy: pool vs silo, tenant context, and noisy neighbor limits

Multi-tenancy serves several customers within one application environment, using dedicated resources, shared resources, or a mixture. Shared resources require tenant-aware authorization and workload controls to protect each customer's data and access to capacity.

Prerequisites

A gateway or load balancer decides which server gets each request (which server gets your request). Queues slow down sharply as utilization nears 100% (why fast servers turn slow under load). A token bucket admits requests at a refill rate with a burst allowance (the "too many requests" error).

Definitions

How does tenant context follow a request?

Follow one request from brand A that tries to read brand B's orders table.

1. The gateway authorizer verifies the token and resolves tenant brand-a. The integration mapping overwrites any X-Tenant-Id the client sent, and the backend accepts traffic only from the gateway, so the header cannot be forged on the way in.
2. The service stores brand-a in request-scoped storage. In .NET that is AsyncLocal<T>; in Node it is AsyncLocalStorage. The value follows the request across await without being passed through every function.
3. Code builds names only through a helper that reads that context, so it produces brand-a_orders. With no tenant in context, the helper throws.
4. Suppose a developer bypasses the helper and hardcodes brand-b_orders. The data call runs with credentials assumed for brand-a, carrying tenant=brand-a as an STS session tag. A role policy that allows only table ARNs matching arn:aws:dynamodb:<region>:<account>:table/${aws:PrincipalTag/tenant}_* denies GetItem on brand-b_orders.

The role policy must reference the session tag; tags alone restrict nothing. Use canonical tenant IDs that exclude the separator and IAM wildcard characters, and match the complete DynamoDB table ARN. Otherwise tenant a would also match tables of a tenant named a_b. The protection also depends on binding credential issuance to the authenticated tenant and removing broader credentials, such as data access on the service's own task role.

In pooled PostgreSQL, row security plays the same role. Enable RLS and define tenant policies for reads and writes. Use an application role without superuser or BYPASSRLS privileges; table owners bypass policies unless RLS is forced. Set transaction-local tenant context (SET LOCAL) when reusing pooled connections. A caller able to change that context can undermine the boundary.

What the OrderServ write-up reports

OrderServ is a Cognizant food-ordering platform used by restaurant brands. According to their write-up on the AWS Architecture Blog (5 November 2026), each brand had its own Amazon ECS cluster, and they consolidated into one shared cluster, with large brands able to keep dedicated capacity:

	Silo (before)	Pool (after)
Clusters for 5 brands	5	1
ECS services	21 × 5 = 105	21
Sidecar containers	315	63
Scale-out time	5 to 7 minutes on Fargate	30 to 60 seconds, EC2 warm pools

The scaling change came from a separate move to EC2 warm pools with instances already prepared; the authors trace the old delay mostly to pulling large images. Their write-up says API Gateway injects tenant context, a .NET tenancy library carries it, and IAM policies back it up. Services that receive tenant context from another service or a queue message should verify that the authenticated sender may act for the supplied tenant.

Worked example: one tenant's surge

Assume one FIFO queue, fixed service times, no drops and no scaling. Eight workers serve 100 req/s each; five tenants send 100 req/s each.

capacity          8 × 100                    = 800 req/s
normal demand     5 × 100                    = 500 req/s   (62.5% busy)
one tenant 10x    4 × 100 + 1,000            = 1,400 req/s
backlog growth    1,400 − 800                = 600 req/s
after a 10 s surge                            = 6,000 queued
wait for a quiet tenant's request then        = 6,000 / 800 = 7.5 s
drain after the surge ends                    = 6,000 / (800 − 500) = 20 s

Every tenant waits, because they share one queue. With an idealized strict limiter of 200 req/s per tenant and the burst allowance used up:

admitted          4 × 100 + 200              = 600 req/s   (75% busy)
rejected (429)    1,000 − 200                = 800 req/s of the surging tenant

OrderServ's write-up describes per-tenant limits set through a Lambda authorizer. For a REST API configured with apiKeySource=AUTHORIZER, the authorizer returns a usage plan's API key as usageIdentifierKey (authorizer output), and that usage plan supplies the throttle. AWS documents these limits as best-effort targets that can be exceeded briefly, so keep bounded queues and concurrency limits on the shared workers too.

Code and our lab run

// labs/multi-tenancy/tenant-context.mjs · node labs/multi-tenancy/tenant-context.mjs
import { AsyncLocalStorage } from "node:async_hooks";
import { setTimeout as sleep } from "node:timers/promises";

const tenantCtx = new AsyncLocalStorage();

function key(kind, name) {
  const id = tenantCtx.getStore()?.tenantId;
  if (!id) throw new Error("no tenant in context"); // fail closed, never a shared name
  return kind === "table" ? `${id}_${name}` : `${id}/${name}`;
}

const log = [];
async function handle(label, delays) {
  for (const [step, ms] of delays.entries()) {
    await sleep(ms); // yield, so the other request runs in between
    log.push(`${label} step ${step + 1}  ${key("table", "orders")}  ${key("cache", "menu")}`);
  }
}

await Promise.all([
  tenantCtx.run({ tenantId: "brand-a" }, () => handle("request A", [5, 20, 5])),
  tenantCtx.run({ tenantId: "brand-b" }, () => handle("request B", [10, 5, 20])),
]);
console.log(log.join("\n"));

try {
  key("table", "orders"); // outside any tenantCtx.run(...)
} catch (e) {
  console.log(`missing context: threw "${e.message}"`);
}

Execution order can vary; each request must retain its own tenant context. Our local run on Node 22 produced 12 resource names across two overlapping requests, with zero cross-tenant names. Calling the helper outside either request threw no tenant in context.

This verifies context propagation only. Whether IAM denies a hardcoded brand-b_orders needs a separate integration test against real scoped credentials.

Failure modes and edge cases

Failure	How you detect it	Fix
Missing context (background job, new thread pool, callback library)	errors from the name helper; requests with no tenant in logs	start jobs inside a context run from the message's verified tenant
Broad credentials (task role can read every table)	IAM Access Analyzer, a test that reads another tenant's resource with the data role	separate data role per tenant session; strip data permissions from the task role
Queue saturation from one tenant	queue depth and p99 by tenant; 429 count by tenant	per-tenant limits, bounded queue, per-tenant concurrency caps
Stale context on pooled DB connections	rows from the wrong tenant in an audit query	SET LOCAL inside each transaction

Trade-offs

Model	Good for	Costs you	Use when
Silo	dedicated capacity, per-tenant tuning, compliance boundaries	one deployment per tenant, idle headroom per tenant	few large tenants, or rules demand separation
Pool	shared headroom, one deployment, onboarding by config	isolation lives in code and policy; noisy neighbors need limits	many tenants with similar needs
Bridge	pooled compute, separated data	two models to operate	data must be apart, compute can be shared

A split into services multiplies silo cost: 21 services became 105 in the source. Monolith vs microservices covers that multiplier.

Common mistakes

In an interview

Clarify tenant count and size spread, then structure by layer: edge, compute, data. Spend most of the time on tenant context and scoped credentials, and finish with per-tenant monitoring.

Practice

Cheat sheet

Term	Meaning
Silo / pool / bridge	dedicated, shared, mixed per layer
Tenant context	verified tenant ID held per request
Session tag	STS value a policy can match on
Backlog growth	demand − capacity
Drain time	backlog / (capacity − normal demand)

Further reading

Source	Good for
OrderServ write-up	the case: counts, tenancy library, warm pools
SaaS Lens models	silo, pool and bridge vocabulary
API Gateway throttling	token bucket, burst, best-effort limits
Authorizer output	returning a usage plan key per tenant
Session tags	tenant-scoped IAM policies
PostgreSQL row security	RLS rules and who bypasses them

Our take

For tenants with similar workload needs, we would consider pooled compute with tenant-scoped access. Test isolation and overload handling before onboarding more tenants, and retain a dedicated deployment option. The test we would write first is the one in step 4: hardcode another tenant's table name and confirm the data role is denied.

The OrderServ details come from their published write-up; we make no claims about their systems beyond it. The noisy-neighbor numbers are an idealized worked example with assumed rates, not a measurement. The lab result is from our own local run.