How to Build Hybrid Multi-Tenant Architecture for Stateful Services on AWS

• Three-tier hierarchy: Tier (tenant grouping) → Cell (AWS account boundary) → Infra Group (VPC + ALB + ECS clusters)
• Route 53 weighted routing distributes traffic across multiple infra groups and cells without client-side DNS changes
• ALB listener rules route by tenant identifier in URL path or HTTP header — maximum ~50 tenants per ALB infra group
• ECS clusters scoped per tenant prevent memory contention in stateful workloads
• AWS PrivateLink established once at tier creation, shared across all tenants — not re-provisioned per tenant
• Onboarding reduced from 52 days to 7 days; infrastructure setup steps cut by 80%

Purpose

Per-tenant AWS account isolation is the gold standard for SaaS data segregation, but it breaks at scale. When 18 clients already require 181 separate targets across 4 regions — with 3% average CPU utilization and 19% memory utilization — provisioning a new account per tenant becomes an engineering tax that compounds with every customer added. This architecture provides strong tenant isolation without per-tenant accounts by moving isolation to the routing and compute layers within a shared account structure.

It is specifically designed for stateful services where large-tenant datasets loaded into memory create noisy-neighbor risks that shared compute cannot mitigate.

Pre-Requisites

• AWS Organization with at least one member account per cell
• Terraform or AWS CDK for infrastructure-as-code (tier and cell resources are provisioned once, tenants added as configuration)
• Amazon ECS (Fargate or EC2 launch type) as the compute layer
• Applications must support a TENANT_ID environment variable for data scoping
• Route 53 hosted zone for the service endpoint domain
• Understanding of ALB target group limits (100 target groups per ALB, 5 per listener rule)

Steps

1. Define the tier structure. A tier is a logical grouping of tenants sharing a performance SLA. Create a Route 53 weighted routing policy for the tier endpoint (e.g., tier-1.us-east-1.example.com) pointing to ALBs across multiple infra groups. Route 53 supports up to 10,000 weighted records per hosted zone.

2. Provision a cell (AWS account). Each cell is an AWS account containing the full tier infrastructure stack. Cells are the horizontal scale-out unit when account-level resource limits (ENIs, VPC endpoints) approach capacity. Cells are provisioned infrequently — typically every 3–4 infra groups.

3. Create an infra group within a cell. An infra group is one VPC, one ALB, and the associated ECS cluster set. Provision VPC, subnets, security groups, and ALB. Establish AWS PrivateLink endpoints at infra group creation — these are shared by all tenants in the group and never re-provisioned per onboarding ($7.30/month per endpoint, negligible when shared across 50 tenants).

4. Configure ALB listener rules for tenant routing. Add a listener rule for each tenant using path-based routing (/tenant-a/*) or header matching forwarding to the tenant’s dedicated target group. Each target group maps to that tenant’s ECS service. Track listener rule count — the 5-target-groups-per-listener-rule constraint means approximately 50 tenants fill an infra group’s routing capacity.

5. Create a dedicated ECS cluster per tenant. Assign a cluster per tenant within the shared VPC. Pass TENANT_ID as an environment variable in the task definition. Configure independent auto-scaling per cluster based on CPU and memory metrics. Tenant-level CloudWatch dimensions (tenant_id, tier_id, region) enable per-tenant observability.

6. Add IAM roles at tier level. Assign IAM roles to ECS task definitions at tier creation. New tenants inherit existing roles — no per-tenant role provisioning. This is the primary mechanism that eliminates the “1 week for IAM” step from the previous 52-day onboarding timeline.

7. Onboard tenants as configuration changes only. For each new tenant: add an ALB listener rule, create an ECS cluster with tenant-scoped task definition, register a target group, and update the Route 53 weighted policy if adding a new infra group. All downstream connectivity (PrivateLink, VPC peering, IAM) is already in place.

8. Scale horizontally by adding infra groups or cells. When an infra group’s ALB approaches the 50-tenant limit, provision a new infra group in the same cell and add it to the Route 53 weighted routing set. When account-level resource limits approach capacity, provision a new cell with a duplicate tier stack.

The Trap

ALB target group limits will catch you earlier than you expect. The AWS default is 100 target groups per ALB and 5 target groups per listener rule. With 1 target group per tenant and standard listener rule configuration, an infra group supports approximately 50 tenants — not 100. Plan your tier capacity with the 50-tenant ceiling in mind, not the nominal ALB limit. Attempting to push past this without adding an infra group creates listener rule conflicts and unpredictable routing behavior under load. See AWS EKS Auto Mode for a related discussion of hidden resource limits in managed AWS services.

Cited References

• AWS Architecture Blog: Building hybrid multi-tenant architecture for stateful services on AWS
• AWS Bedrock AgentCore: Managed Agentic AI Runtime Architecture
• AWS ALB listener rule documentation
• AWS Route 53 weighted routing policy

Photo by panumas nikhomkhai on Pexels

Source: AWS Architecture Blog

Content created with AI assistance and reviewed for accuracy.