Every B2B SaaS platform begins with an innocent decision: How should we partition customer data?
In the early days of Atlassian cloud, the architecture relied on physical sharding across hundreds of individual relational databases. Over years of explosive customer adoption, this grew into a sprawl of approximately 750 sharded PostgreSQL database clusters.
At that scale, systems stop breaking because of bugs; they break because of physics, operating system ceilings, and database topology limits.
Here is what broke, why 750 sharded database clusters became an existential operational bottleneck, and how the platform migrated to 16 distributed-SQL clusters to power the next generation of cloud growth.
---
1. The Multi-Tenancy Spectrum: Silo vs. Pool vs. Bridge
Before analyzing the failure modes, it is critical to understand the three primary multi-tenancy models:
| Dimension | Silo Model (Isolated DB) | Pool Model (Shared DB + Tables) | Hybrid Bridge Model |
| :--- | :--- | :--- | :--- |
| Isolation | Absolute physical isolation | Logical (tenant_id discriminator) | Distributed SQL with tenant ranges |
| Infrastructure Cost | Extremely high (idle DB compute) | Very low (max resource utilization) | Balanced (elastic horizontal nodes) |
| Operational Overhead | Catastrophic (thousands of clusters) | Manageable (single schema evolution) | Low (declarative schema DDL) |
| Noisy Neighbors | None | High risk without strict rate limiting | Controlled via locality & range leases |
Atlassian’s initial cloud design leaned heavily toward the Silo model: customers were partitioned across ~750 distinct PostgreSQL instances. While this provided strong theoretical isolation, it rapidly hit three catastrophic failure modes.
---
2. Failure Mode 1: Connection Pool Exhaustion
PostgreSQL utilizes a process-based connection model: every connected client fork consumes roughly 5MB to 10MB of server memory, plus dedicated kernel thread scheduling overhead.
As Jira and Confluence expanded into dozens of microservices, each service required a database connection pool to talk to every customer shard:
$$ ext{Total Connections} = ext{Microservices} imes ext{App Instances} imes ext{Pool Size} imes ext{Tenant Shards}$$
Even with connection pooling proxies like PgBouncer running in transaction mode, the sheer cardinality of 750 database endpoints multiplied across hundreds of Kubernetes application pods saturated maximum connection limits.
Database instances were spending more CPU time context-switching between idle connection processes than executing relational queries.
---
3. Failure Mode 2: The Noisy Neighbor Catastrophe
In a sharded architecture, hash-based placement creates severe resource skew. When a global Fortune 50 enterprise tenant—with 500,000 active users and automated CI/CD webhooks firing 200 times per second—lands on Shard 42 alongside 300 smaller startup tenants, chaos ensues.
The large tenant’s queries thrash PostgreSQL’s shared_buffers, evicting the smaller tenants' cached indexes. Small startups suddenly experience query latency spikes from 12ms to 4,000ms through no fault of their own.
Rebalancing an oversized tenant off Shard 42 to a dedicated cluster required: Taking an exclusive table lock or coordinating asynchronous logical replication. Re-syncing sequence numbers and foreign keys. * Hours of high-stress operational maintenance windows.
---
4. The Solution: Consolidating to 16 Distributed-SQL Clusters
To escape the 750-cluster sharding trap, the platform embarked on a monumental architectural migration: consolidating the fragmented fleet into 16 distributed-SQL clusters utilizing Multi-Raft consensus.
[ Multi-Tenant Traffic ]
│
▼
[ Distributed SQL Cluster ]
├── Range 1: Tenant A (Frankfurt, EU) -> Raft Group 1
├── Range 2: Tenant B (Virginia, US) -> Raft Group 2
└── Range 3: Tenant C (Sydney, AU) -> Raft Group 3#### Key Architectural Benefits:
1. Elastic Range Splitting: Data is divided into contiguous 64MB storage ranges. When a large tenant’s data grows, the cluster automatically splits the range and relocates it across storage nodes without human intervention or downtime. 2. Tenant Locality Pinning: Distributed SQL allows geo-partitioning at the table row level. Data belonging to a European customer is pinned to storage nodes in Frankfurt to satisfy GDPR data residency mandates, while US data remains in Virginia. 3. Unified Connection Pooling: Instead of maintaining connection pools to 750 separate PostgreSQL endpoints, microservices connect to any local node in the distributed cluster, which routes queries internally to the appropriate Raft range leaseholder.
---
5. Defensive Isolation: Row-Level Security (RLS)
In a pooled distributed architecture, accidental data leaks between tenants must be guarded against with absolute mathematical certainty. Relying on application developers to remember WHERE tenant_id = ? in every query is a recipe for disaster.
We enforce PostgreSQL Row-Level Security (RLS) as a defense-in-depth gate at the database engine level:
-- 1. Enable RLS on core domain table
ALTER TABLE issues ENABLE ROW LEVEL SECURITY;
-- 2. Create unbypassable tenant isolation policy
CREATE POLICY tenant_isolation_policy ON issues
AS RESTRICTIVE
USING (tenant_id = current_setting('app.current_tenant_id', true)::uuid);Before executing any query, the database connection pool sets the session context:
func (db *TenantDB) WithTenant(ctx context.Context, tenantID string, fn func(tx *sql.Tx) error) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
// Enforce tenant boundary in database session variable
_, err = tx.ExecContext(ctx, "SET LOCAL app.current_tenant_id = $1", tenantID)
if err != nil {
return err
}
if err := fn(tx); err != nil {
return err
}
return tx.Commit()
}If a developer writes SELECT * FROM issues;, the database engine automatically appends the RLS filter. If no tenant context is set, the query returns zero rows.
---
Summary
Scaling multi-tenancy from thousands to millions of organizations requires moving beyond static physical sharding. By replacing 750 brittle PostgreSQL clusters with elastic distributed-SQL clusters and enforcing Row-Level Security, platforms can achieve true cloud elasticity, compliance with global data residency laws, and ironclad tenant isolation.