Designing a Database-as-a-Service Platform: Tenant Provisioning, Compute-Storage Separation, and Connection Routing at Scale

Running a database for yourself is hard. Running a database for ten thousand other people is a different category of problem. When you build a Database-as-a-Service platform, every design decision you make for a single-tenant system gets multiplied by your tenant count, and the failure modes that were theoretical edge cases become daily operational reality.

This article walks through how modern DBaaS platforms are architected under the hood. The core challenges are: provisioning tenant databases and managing their lifecycle, separating compute from storage so databases can scale and sleep independently, routing connections across thousands of tenant endpoints without melting the connection limit of the underlying database engine, keeping tenants isolated from each other’s resource consumption, and recovering from failures at a granularity that satisfies real SLA requirements.

Tenant Provisioning and Lifecycle Management

The simplest version of tenant provisioning is calling CREATE DATABASE and handing back a connection string. This works until it does not, which is around the point you have a few hundred tenants and need to track metadata, enforce quotas, handle failures mid-provisioning, and deactivate or delete databases on plan changes.

A robust provisioning system models the tenant database as a resource with a lifecycle: pending, provisioning, active, suspended, deprovisioning, deleted. Each state transition is an operation that may span multiple steps and external systems. That means it needs to be a workflow, not a function call.

type DatabaseLifecycleState =
  | "pending"
  | "provisioning"
  | "active"
  | "suspended"
  | "deprovisioning"
  | "deleted";

interface TenantDatabase {
  id: string;
  tenantId: string;
  region: string;
  storageNodeId: string;
  computeNodeId: string | null; // null when suspended (serverless)
  connectionEndpoint: string;
  state: DatabaseLifecycleState;
  planTier: "shared" | "dedicated-compute" | "serverless";
  storageGb: number;
  provisionedAt: Date | null;
  lastActiveAt: Date | null;
}

The provisioning workflow itself has distinct phases. First, a record is created in the control plane with state pending. Then storage is allocated on the appropriate storage node. Then a compute instance is attached and configured. Then the database engine is initialized, the schema is bootstrapped, and the endpoint is registered in the routing layer. Only after all of that does the state flip to active.

Each phase needs to be idempotent. If provisioning fails after storage allocation but before compute attachment, the retry must not allocate a second storage volume. You track phase completion in the control plane record and skip completed phases on retry:

interface ProvisioningCheckpoint {
  databaseId: string;
  storageAllocated: boolean;
  computeAttached: boolean;
  engineInitialized: boolean;
  endpointRegistered: boolean;
  completedAt: Date | null;
}

async function provisionDatabase(
  db: TenantDatabase,
  checkpoint: ProvisioningCheckpoint
): Promise<void> {
  if (!checkpoint.storageAllocated) {
    await allocateStorageVolume(db.storageNodeId, db.id, db.storageGb);
    await updateCheckpoint(db.id, { storageAllocated: true });
  }

  if (!checkpoint.computeAttached) {
    const computeNode = await selectComputeNode(db.region, db.planTier);
    await attachCompute(computeNode.id, db.id);
    await updateCheckpoint(db.id, { computeAttached: true, computeNodeId: computeNode.id });
  }

  if (!checkpoint.engineInitialized) {
    await initializeDatabaseEngine(db.id);
    await updateCheckpoint(db.id, { engineInitialized: true });
  }

  if (!checkpoint.endpointRegistered) {
    await registerRoutingEndpoint(db.id, db.connectionEndpoint);
    await updateCheckpoint(db.id, { endpointRegistered: true });
  }

  await markDatabaseActive(db.id);
}

Deprovisioning follows the same pattern in reverse, with the critical constraint that storage is the last thing you delete. Compute is detached first, then the routing entry is removed, then after a configurable retention window the storage volume is purged. That window is your last line of defense against accidental deletion: if a tenant asks for their data back within 30 days, you can still restore it.

Compute-Storage Separation: The Enabling Architecture

Traditional database deployments couple compute and storage on the same machine. This made sense when the I/O cost of crossing a network boundary was prohibitive. Modern high-bandwidth networks with NVMe-over-Fabrics and similar protocols have changed that calculus enough that separating the two layers is now the dominant architecture for cloud-native databases.

In a compute-storage separated system, the storage layer holds the authoritative copy of the data and the write-ahead log. Compute nodes are stateless from the storage perspective: they can crash and restart without losing any committed data, because all durability is delegated to the storage layer. This separation enables several things that matter for a DBaaS:

Scale-to-zero. If a tenant has not queried their database in 5 minutes, you can terminate their compute node and stop paying for it. When the next query arrives, you spin up a new compute node, attach it to the existing storage volume, and resume. The cold-start latency is a function of how fast you can attach and warm the compute, not how long database initialization takes from scratch. Serverless databases are built on this capability.

Independent scaling. Compute and storage scale on separate axes. A tenant with a large dataset but low query rate needs more storage, not more compute. A tenant running analytics queries needs more CPU, not more disk. Coupling these resources together forces you to over-provision one to get enough of the other.

Read replicas without storage duplication. When storage is shared, adding a read replica means attaching another compute node to the same storage volume. You pay for compute, not for another copy of the data. Replication lag in this model is not about log shipping: it is about how quickly the replica’s page cache catches up with writes committed to shared storage.

The storage layer itself is typically a distributed log-structured system. Writes are appended to a WAL segment. The WAL is replicated across availability zones for durability. Compute nodes read WAL entries to keep their buffer pool current. When a compute node cold-starts, it replays the WAL from a recent checkpoint:

interface WalSegment {
  segmentId: string;
  databaseId: string;
  startLsn: bigint; // log sequence number
  endLsn: bigint;
  sizeBytes: number;
  durabilityQuorum: number; // number of AZ copies confirmed
  createdAt: Date;
}

interface ComputeAttachRequest {
  databaseId: string;
  computeNodeId: string;
  resumeFromLsn: bigint; // replay WAL from this point
}

This architecture has one significant tradeoff: read-write latency increases because every write must cross the network boundary to the storage layer. For write-heavy workloads with tight latency requirements, the overhead is measurable. The mitigation is colocating compute and storage in the same availability zone (reducing round-trip latency to sub-millisecond on modern fabric networks) and batching WAL writes to amortize the cost.

Connection Routing at Scale

Postgres has a fundamental limitation that every DBaaS operator discovers fast: each client connection spawns a backend process on the server. At a few hundred connections, memory pressure builds. At a few thousand, the OS process table becomes a bottleneck. In a multi-tenant platform serving thousands of databases, the naive approach of direct client connections is immediately untenable.

The solution is a connection routing and pooling layer that sits between clients and the actual database backends. This layer has two jobs: route connections to the correct tenant database, and pool connections so that N active clients share M backend connections where M is much smaller than N.

For routing, the simplest approach encodes the tenant identifier in the connection hostname. A client connecting to tenant-abc.db.example.com is routed by the connection layer to the database registered under that identifier. The routing layer maintains a table mapping hostnames to backend endpoints, updated in real-time as databases are provisioned and deprovisioned:

interface RoutingEntry {
  hostname: string; // tenant-abc.db.example.com
  databaseId: string;
  backendHost: string;
  backendPort: number;
  poolConfig: PoolConfig;
  state: "active" | "suspended" | "draining";
}

interface PoolConfig {
  poolMode: "transaction" | "session" | "statement";
  maxServerConnections: number;
  maxClientConnections: number;
  reservedConnections: number; // kept for admin/monitoring use
  connectTimeoutMs: number;
  idleTimeoutMs: number;
}

Transaction-mode pooling is the highest-leverage configuration for a DBaaS. In transaction mode, a backend server connection is held only for the duration of a transaction, then returned to the pool. A single server connection can serve hundreds of clients as long as they are not all in long-running transactions simultaneously. The tradeoff is that session-level state (temporary tables, prepared statements, SET variables) does not survive between transactions.

For tenants with serverless databases (compute scaled to zero), the routing layer must handle the cold-start path. When a connection arrives for a suspended database, the router must trigger compute reattachment before proxying the connection. This means the routing layer needs to know the lifecycle state of every database and have the authority to trigger state transitions:

async function handleIncomingConnection(
  conn: IncomingConnection,
  router: RoutingTable
): Promise<void> {
  const entry = await router.lookup(conn.hostname);

  if (!entry) {
    throw new Error(`No database found for hostname: ${conn.hostname}`);
  }

  if (entry.state === "suspended") {
    // Cold start: wake the compute, wait for attach, then proxy
    await triggerComputeWakeup(entry.databaseId);
    await waitForState(entry.databaseId, "active", { timeoutMs: 5000 });
    await router.refreshEntry(entry.databaseId);
  }

  if (entry.state === "draining") {
    // Database is being deprovisioned, reject gracefully
    conn.reject("Database is no longer available");
    return;
  }

  await proxyToBackend(conn, entry.backendHost, entry.backendPort);
}

The routing layer is also where you enforce connection limits per tenant. A tenant on a starter plan should not be able to exhaust shared pool capacity for other tenants. You track active client connections and active server connections per database and reject connection requests that would exceed the plan limit.

Backup Orchestration and Point-in-Time Recovery

For a DBaaS, backup is not optional and it is not “run pg_dump once a night.” The baseline expectation is continuous backup with point-in-time recovery (PITR) to any second within the retention window.

In a compute-storage separated system, continuous backup is partially solved by the storage layer. Because all writes flow through the WAL and the WAL is durably replicated, you already have a continuous log of every change. PITR is a matter of replaying that log from a base snapshot to a target LSN.

The implementation has three components:

Base snapshots. Taken on a schedule (typically every 24 hours), these are full copies of the storage volume at a specific LSN. They establish the starting point for PITR. Snapshots are uploaded to object storage (S3-compatible) and indexed in the control plane with their LSN and timestamp.

Continuous WAL archiving. WAL segments are streamed to object storage in near-real-time as they are sealed. The archiver tracks the last successfully uploaded LSN. The gap between the latest sealed WAL segment and the latest archived segment is your recovery point objective exposure.

Restore orchestration. When a tenant requests PITR to timestamp T, the restore process finds the most recent base snapshot before T, provisions a fresh compute node with that snapshot, and replays WAL segments from the snapshot LSN up to the LSN corresponding to T.

interface BackupCatalog {
  databaseId: string;
  snapshots: BaseSnapshot[];
  walArchiveStartLsn: bigint;
  walArchiveEndLsn: bigint; // latest durably archived
  oldestRestorableTimestamp: Date;
  retentionDays: number;
}

interface BaseSnapshot {
  snapshotId: string;
  databaseId: string;
  atLsn: bigint;
  atTimestamp: Date;
  storageUri: string; // s3://bucket/snapshots/...
  sizeBytes: number;
}

async function initiatePointInTimeRestore(
  databaseId: string,
  targetTimestamp: Date,
  catalog: BackupCatalog
): Promise<string> {
  // Find the base snapshot immediately before the target
  const baseSnapshot = catalog.snapshots
    .filter((s) => s.atTimestamp <= targetTimestamp)
    .sort((a, b) => Number(b.atLsn - a.atLsn))[0];

  if (!baseSnapshot) {
    throw new Error("No base snapshot found before target timestamp");
  }

  const targetLsn = await timestampToLsn(databaseId, targetTimestamp);

  // Spin up a new database, do not overwrite the source
  const restoredDatabaseId = await createDatabaseFromSnapshot(
    baseSnapshot.storageUri,
    baseSnapshot.atLsn
  );

  await replayWalSegments(restoredDatabaseId, baseSnapshot.atLsn, targetLsn);

  return restoredDatabaseId;
}

One detail worth getting right: PITR restores create a new database rather than overwriting the existing one. If a tenant requests PITR to recover from accidental data deletion, you want to give them a restored copy to inspect before they commit to replacing their primary. Overwriting the primary in-place is a footgun.

Resource Isolation and Noisy Neighbor Prevention

When thousands of tenants share infrastructure, any one of them can generate load that degrades the others. This is the noisy neighbor problem, and it manifests at every layer: storage I/O, network bandwidth, CPU on shared compute nodes, and connection pool exhaustion.

The first line of defense is the isolation model. Tenants on dedicated compute plans do not share a server with others, so their CPU and memory is not contested. But storage is still shared in many architectures, so I/O isolation requires explicit rate limiting.

At the storage layer, you enforce I/O operations per second (IOPS) limits per tenant volume. Each storage write from a compute node is tagged with the tenant’s database ID. The storage layer tracks rolling window IOPS consumption per tenant and applies backpressure when limits are exceeded. Backpressure means delaying the write, not dropping it: the semantics of the operation must be preserved, just at reduced throughput.

At the connection routing layer, the per-tenant pool limits described earlier enforce connection isolation. A tenant on a starter plan with a maxServerConnections of 10 cannot starve shared connection capacity even if they open 10,000 client connections simultaneously: 10 server connections is the ceiling, and the pool queues or rejects the rest.

For tenants on shared compute (the lowest tier, where multiple tenant databases run on the same compute node), CPU enforcement is the hardest problem. You can use cgroups to allocate a CPU budget per process, but the granularity and overhead of cgroup accounting at this scale requires careful tuning. An alternative approach is workload admission control at the query level: queries from tenants that have exceeded their compute budget are queued with lower priority or rejected with an informative error.

A practical signal for noisy neighbor detection is tracking p99 query latency per tenant and alerting when one tenant’s queries are consistently slower than baseline, which often indicates resource contention from a neighbor:

interface TenantResourceMetrics {
  databaseId: string;
  windowStartMs: number;
  iopsConsumed: number;
  iopsLimit: number;
  activeServerConnections: number;
  connectionLimit: number;
  cpuMillicoresConsumed: number;
  cpuMillicoresLimit: number;
  queryLatencyP99Ms: number;
}

function isNoisyNeighborCandidate(metrics: TenantResourceMetrics): boolean {
  const iopsRatio = metrics.iopsConsumed / metrics.iopsLimit;
  const cpuRatio = metrics.cpuMillicoresConsumed / metrics.cpuMillicoresLimit;
  return iopsRatio > 0.85 || cpuRatio > 0.85;
}

Tenants consistently hitting their resource ceilings are candidates for plan upgrades or for migration to a less-contended storage or compute node.

Deployment Model Tradeoffs

Modern DBaaS platforms offer multiple deployment models, each with a different tradeoff surface. Understanding the tradeoffs is necessary for deciding which models to offer and which to prioritize.

Model	Cold-Start Latency	Cost at Rest	Max Throughput	Isolation	Complexity
Shared compute (pooled)	None	Very low	Limited by pool	Weakest	Low
Dedicated compute	None	High (always on)	High	Strong	Medium
Serverless (scale-to-zero)	500ms to 5s	Near zero	Scales with demand	Strong	Highest
Shared-nothing (per-tenant server)	None	Very high	Highest	Complete	Very high

Shared compute, pooled tenants. Multiple tenant databases run on the same compute node. You get the highest density (lowest cost per tenant) but the weakest isolation. CPU and memory are not hard-isolated. Good for development and starter tiers where traffic is negligible.

Dedicated compute. Each tenant gets a compute node that is not shared. Storage may still be on a shared storage cluster, but compute resources are guaranteed. Suitable for production workloads with consistent traffic patterns that do not benefit from scale-to-zero economics.

Serverless (compute scale-to-zero). Compute is provisioned on-demand and released after an inactivity window. The cold-start path (attaching compute to storage, warming the buffer pool, registering with the router) determines the quality of the user experience. Minimizing cold-start time is an engineering discipline unto itself: pre-warming compute nodes at prediction time (based on tenant usage patterns), keeping WAL replay short by taking frequent checkpoints, and maintaining a warm compute pool for fast allocation all contribute.

Shared-nothing. Each tenant gets their own database server and their own storage, fully isolated at every layer. This is the most expensive model and the most complex to automate at scale, but it is the only model that can satisfy strict compliance requirements (data residency, single-tenant guarantees for HIPAA or FedRAMP workloads). Cloud providers offering dedicated instances for enterprise customers are effectively offering shared-nothing.

Production Considerations

Control plane availability is more critical than data plane availability. If the control plane goes down, tenants cannot provision new databases, cannot scale, and your on-call team cannot manage incidents. The data plane (actual database connections) is often more fault-tolerant because it is stateless routing to backends. Treat the control plane as a tier-0 service with its own dedicated infrastructure.

Connection routing tables need cache invalidation with strong consistency guarantees. If you cache routing entries on router nodes and a database is deprovisioned, a stale cache entry will route connections to a backend that no longer exists. Use short TTLs or push-based invalidation from the control plane when state transitions occur.

WAL archiving lag is a real RPO risk. Monitor the gap between the last sealed WAL segment and the last archived segment continuously. If archiving falls behind due to object storage throttling or network issues, your actual RPO degrades silently. Alert on archiving lag, not just on archiving failures.

Cold-start paths need circuit breakers. If the compute wakeup service is degraded, every connection attempt to a suspended database will hang waiting for wakeup. The router needs a timeout and a graceful failure path: reject the connection with a specific error after N seconds rather than holding it open indefinitely.

Plan migrations (upgrades and downgrades) require careful orchestration. Moving a tenant from shared compute to dedicated compute involves detaching from the shared pool, attaching to a dedicated node, and updating the routing entry, all while the tenant may have active connections. The safest approach is to provision the new environment, verify it is healthy, drain connections from the old environment with a brief connection hold, then flip the routing entry and release the old resources. The drain window is typically a few seconds for transaction-mode pools.

Storage compaction runs at the platform level, not the tenant level. LSM-based storage engines need background compaction to reclaim space and maintain read performance. When you are running compaction across thousands of tenant volumes simultaneously, the I/O amplification can saturate your storage cluster. Stagger compaction schedules across tenants and throttle compaction I/O as a lower-priority workload class.

Building a database platform is one of the few engineering problems where the operational discipline required exceeds the implementation complexity. The core systems are straightforward to implement in isolation. What is hard is making them compose correctly across ten thousand tenants, under failure modes that only appear at that scale, with the SLA constraints that paying customers rightfully expect.